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Abstract 

To  properly  manage  the  Installation  Restoration  Program 
(IRP)  in  the  future,  the  Air  Force  (AF)  needs  a  method  to 
assess  progress  based  on  improvements  in  environmental 
conditions.  An  integral  part  of  evaluating  progress  toward 
protecting  human  health  and  the  environment  is  the 
assessment  of  risk  reduction  as  a  result  of  cleanup  actions. 

The  AF  currently  uses  the  Defense  Priority  Model  (DPM) 
to  assist  in  setting  priorities  for  funding  remedial  actions 
at  IRP  sites.  The  DPM  provides  a  numerical  score  for  IRP 
sites  representing  the  relative  potential  risk  based  on  the 
environmental  conditions  at  a  site  before  remedial  actions 
are  taken.  This  study  investigates  using  the  DPM  to  provide 
an  indicator  of  progress  based  on  improvements  in 
environmental  conditions  at  IRP  sites. 

Rescoring  the  DPM  to  represent  site  conditions  after 
remedial  actions  shows  a  reduction  in  relative  risk  as 
contaminant  levels  and  transport  potential  are  changed. 

A  multi-criteria  decision  modeling  techniques  was  used  to 
combine  the  relative  risk  data  with  other  administrative 
measures  of  progress  currently  used  by  the  AF.  The  results 
provide  a  method  to  rank  the  accomplishments  at  IRP  sites. 
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RISK  REDUCTION  AS  A  CRITERION  FOR  MEASURING  PROGRESS 


OF  THE  INSTALLATION  RESTORATION  PROGRAM 


I.  Introduction 


General  Issue 


The  Air  Force  (AF)  has  spent  1.7  billion  dollars  over 
the  last  eight  years  to  address  the  problems  of  sites 
contaminated  with  hazardous  substances  An  additional  seven 
billion  dollars  will  be  required  fc  remedial  efforts  at 
thousands  of  sites  over  the  next  decade  (20) .  To  justify 
and  properly  manage  resources  in  the  future  the  AF  needs  a 
method  to  assess  progress  based  on  improvements  in 
environmental  conditions  at  contaminated  sites. 

The  AF  currently  uses  administrative  milestones,  such 
as  number  of  sites  closed,  to  measure  progress  toward 
cleaning  up  contaminated  sites.  These  milestones  do  not 
accurately  reflect  what  has  been  accomplished  to  improve  the 
environmental  condition  at  contaminated  sites  (3:144).  Don 
R.  Clay,  the  Environmental  Protection  Agency's  (EPA)  top 
official  responsible  for  the  national  "Superfund"  cleanup 
program,  states: 

Attention  has  been  focused  on  the  number  of  sites 
deleted  from  the  National  Priorities  List  (NPL)  as  the 
only  measure  of  Superfund  accomplishment.  Evaluating 
Superfund's  success  by  tallying  site  deletions  is  not 
only  disappointing,  it  is  inherently  misleading . Success 
for  Superfund  is  more  appropriately  measured  in  terms 
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of  the  successive,  interim  actions  taken  to  control  the 
worst  problems  at  the  worst  sites.  (3:144) 

An  integral  part  of  evaluating  progress  toward 

protecting  human  health  and  the  environment  is  the 

assessment  of  risk  reduction  (6:1333).  Without  assessing 

risk  reduction  associated  with  remedial  actions  there  is 

no  basis  for  measuring  improvements  in  environment 

conditions  (7:1535). 

Installation  Restoration  Program  (IRP) .  The  AF 
established  the  Installation  Restoration  Program  (IRP)  to 
address  the  problems  of  sites  contaminated  by  hazardous 
waste.  The  AF  is  required  to  comply  with  the  Comprehensive 
Environmental  Response,  Compensation,  and  Liability  Act  of 
1980  (CERCLA)  and  the  Superfund  Amendments  and 
Reauthorization  Act  of  1986  (SARA) .  These  laws  are 
popularly  known  as  the  Superfund  Program.  Section  211  of 
SARA,  known  as  the  Defense  Environmental  Restoration  Program 
(DERP) ,  specifically  states  the  role  of  Department  of 
Defense  (DOD)  facilities  in  complying  with  CERCLA/SARA. 

The  AF  IRP  is  a  subcomponent  of  DERP.  The  objectives  of 
the  IRP  are  consistent  with  the  DERP  as  stated  in  SARA 
Section  211  (5:59,60): 

The  identification,  investigation,  research  and 
development,  and  cleanup  of  hazardous  substances, 
pollutants,  and  contaminants.  (24:1670) 

The  Defense  Environmental  Restoration  Account  (DERA) 

was  established  to  support  the  DERP  and  its  subcomponents, 

such  as  the  Air  Force  IRP.  DERA  funds  are  appropriated 
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annually  a. id  are  centrally  managed  by  the  Office  of  the 
Deputy  Assistant  Secretary  of  Defense  (Environment) 

(OSD(E)).  CERCLA/SARA  requires  an  annual  report  to 
Congress  on  the  expenditure  of  DERA  funds  and  the  progress 
of  the  DERP. 

The  only  measures  of  progress  used  thus  far  in  the  IRP 
have  been  based  on  administrative  or  legal  milestones. 

There  is  curren  ly  no  method  for  evaluating  improvement  of 
environmental  conditions  as  a  result  of  remedial  actions  at 
IRP  sites.  To  justify  and  manage  the  resources  for  the  IRP, 
the  AF  needs  a  method  to  assess  progress  based  on  tangible 
environmental  improvements.  Because  of  the  high  level  of 
interest  and  oversight,  it  is  important  tor  the  AF  to  show 
progress  toward  restoring  the  environment  and  eliminating 
the  threat  posed  by  the  worst  sites. 

Research  Problem 

In  order  to  develop  a  method  to  assess  progress  based 
on  improvements  in  environmental  conditions  at  contaminated 
sites  two  research  problems  will  be  addressed.  First,  a 
method  to  quantify  risk  reduction  at  IRP  sites  needs  to  be 
developed.  Second,  a  method  for  combining  risk  reduction 
data  with  other  factors  (such  as  administrative  milestones) 
to  measure  progress  needs  to  be  developed. 

Research  Objectives 

The  purpose  of  this  study  is  to  develop  a  method  to  use 
risk  reduction  as  a  criterion  to  assess  progress  for  the 
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IRP.  This  study  investigates  the  application  of  current 
risk  assessment  techniques  to  quantify  reduction  in 
environmental  risk  at  IRP  sites  resulting  from  remedial 
actions.  Once  reduction  in  risk  can  be  quantified,  a  method 
will  be  developed  to  consider  risk  reduction  along  with 
other  factors  to  provide  a  more  meaningful  measure  of 
progress  for  the  IRP. 

Definition  of  Progress 

For  this  research,  progress  is  defined  as  the 
measurement  of  accomplishment  toward  improving  environmental 
conditions  resulting  from  actions  taken  as  part  of  the  IRP . 
Measurements  of  progress  are  needed  to  compare  the  relative 
accomplishments  of  one  project  or  program  to  another.  This 
allows  the  popular  use  of  a  scorecard  to  compare  the 
accomplishments  made  by  various  organizations  within  the  AF . 
Measurements  of  progress  are  applicable  for  a  site  by  site 
comparisons  or  b>  base,  MAJCOM,  or  service  agency. 

Application  and  Limitation 

This  research  addresses  the  measurement  of  progress 
from  the  point  when  decisions  are  made  for  remedial  actions 
(or  site  closures)  at  IRP  sites.  The  intent  of  the  research 
is  to  develop  a  method  to  compare  the  accomplishments  of 
actions  taken  at  IRP  sites.  Extensive  remedial  actions 
necessary  at  the  worst  sites  will  be  compared  to  lesser 
actions  taken  to  close  sites  where  little  or  no 
environmental  threat  exists. 
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Existing  methods  for  measuring  risk  and  multi-criteria 
evaluation  of  alternatives  will  be  used  to  demonstrate  how 
they  can  be  used  to  assess  progress.  It  is  not  the  focus  of 
this  research  to  evaluate  or  validate  the  methods  used  to 
assess  risk  at  IRP  sites.  Also,  it  is  not  the  intent  of 
this  research  to  determine  that  a  particular  method  for 
multi-criteria  evaluation  of  alternatives  is  more  suitable 
than  others.  A  wide  variety  of  methods  to  measure  risk  and 
evaluate  alternatives  may  be  applicable  for  the  methods 
developed  in  this  study  to  measure  progress. 
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II.  Literature  Review 


"The  lack  of  assessment  of  risk  reduction  is  a  major 
weakness  in  the  present  decision  making  process  [for  the 
Superfund  Program]"  (6:1333).  The  objective  of  this 
research  is  twofold:  develop  a  method  to  assess  reduction  in 
risk  at  IRP  sites  and  to  rank  or  prioritize  sites  based  on 
progress.  First,  a  method  to  evaluate  reduction  in  risk 
needs  to  be  developed.  The  following  provides  a  brief 
discussion  of  how  risk  is  measured  and  several  methods 
used  to  evaluate  environmental  risk  at  hazardous  waste 
disposal  sites. 

The  second  objective  is  to  use  risk  reduction  data 
along  with  other  factors,  such  as  administrative  milestones, 
to  assess  the  accomplishments  of  the  IRP.  Various  multi¬ 
criteria  evaluation  techniques  have  been  used  in  many 
applications  to  rank  alternatives,  where  each  of  the 
alternatives  have  many  factors  to  consider.  These 
techniques  can  be  used  for  ranking  the  accomplishments  of 
IRP  actions  based  on  several  factors  that  indicate  progress. 
This  chapter  provides  a  discussion  of  the  literature 
reviewed  on  multi-criteria  evaluation  methods  and  various 
Multi-Criteria  Decision  Models  (MCDM)  that  are  available. 

Measures  of  Risk 

Risk  assessment  provides  the  scientific  basis  necessary 
for  making  environmental  management  decisions  (17:192). 
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Risk  assessment  is  broadly  defined  by  Cohrssen  and  Covello 
as  "the  technical  assessment  of  the  nature  of  risk"  (4:1). 
There  are  a  variety  of  purposes,  applications,  and  methods 
for  assessing  the  risks  to  human  health  and  the  environment 
associated  with  IRP  sites.  The  various  applications  include 
both  qualitative  and  quantitative  methods  to  describe  risk 
(19:18).  There  are  two  types  of  quantitative  measures  of 
risk  commonly  used  in  the  IRP  and  Superfund,  they  are 
quantitative  risk  assessments  (for  assessing  excess  risk) 
and  relative  risk  models  (16) . 

Quantitative  Risk  Assessment.  Quantitative  risk 
assessments  are  used  to  estimate  the  excess  risk  or  adverse 
impacts  of  contaminants  to  exposed  populations  and  the 
environment  (19:18).  Risk  assessments  involve  four  steps: 
hazard  identification,  dose-response  assessment,  exposure 
assessment,  and  risk  characterization.  The  results  of  risk 
assessments  are  expressed  in  terms  of  excess  risk  or  the 
probability  and  severity  of  an  adverse  response  to  an 
exposure  to  a  contaminant  (16)  , 

There  are  defined  EPA  guidelines  for  assessing  excess 
risk.  However,  at  each  site,  typically  a  wide  variety  of 
inferences  can  be  made  from  the  data  available  and  the  EPA 
guidelines.  The  methods  currently  used  for  quantitative 
risk  assessment  have  been  highly  controversial  within  the 
field  of  environmental  management.  Quantitative  risk 
assessments  require  condensing  highly  uncertain,  frequently 
conflicting,  and  often  ambiguous  data.  The  results  are 
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inferred  from  data  extrapolated  well  beyond  what  can 
actually  be  measured  to  assign  a  value  that  represents  the 
excess  risk  to  a  certain  population  (17:191).  The 
controversial  nature  of  the  science  of  risk  assessment  as 
well  as  local  regulatory,  political,  and  social  influences 
has  make  it  difficult  to  use  a  consistent  method  for 
quantitative  risk  assessment.  No  standard  method  has 
evolved  for  assessing  the  excess  risk  posed  by  environmental 
hazards . 

Relative  Risk  Assessment.  The  CERCLA/SARA  legislation 
required  EPA  to  develop  a  system  to  assess  the  relative 
degree  of  risk  to  human  health  and  the  environment  at 
potentially  contaminated  sites.  The  EPA  developed  the 
Hazard  Ranking  System  (HRS)  to  provide  a  quantitative 
assessment  of  the  relative  threat  posed  at  potential 
Superfund  sites  (27:1). 

The  HRS  is  used  as  a  screening  tool  to  identify  those 
sites  that  represent  the  highest  priority  for  further 
investigation  and  possible  cleanup  under  CERCLA.  Its 
purpose  is  not  to  fully  characterize  the  source  and 
extent  of  the  contamination.  Rather,  its  purpose  is  to 
evaluate  the  potential  of  uncontrolled  hazardous 
substances  to  cause  damage  to  human  health  or  to  the 
environment.  Uniform  application  of  the  HRS  nationwide 
enables  EPA  to  evaluate  sites  relative  to  each  other 
with  respect  to  actual  or  potential  hazards.  (27:2-3) 

The  HRS  uses  data  generated  during  the  Preliminary 

Assessment  (PA)  and  Site  Inspection  (SI)  phases  of  the 

CERCIiA  remedial  action  process.  The  HRS  score  is  used 

to  determine  if  a  site  is  placed  c  the  NPL  for  further 

investigation  and  cleanup  in  the  Superfund  program  (27:1). 
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The  HRS  is  designed  to  be  a  simple  numerical  model  that 
assigns  a  score  from  1  to  100  based  on: 

1.  The  likelihood  that  a  site  has  released  or  has 
the  potential  to  release  contaminants  to  the 
environment . 

2.  The  characteristics  of  the  waste  (toxicity  and 
waste  quantity) . 

3.  The  people  or  sensitive  environments  affected  by 
the  release.  (27:1) 

Other  federal  agencies  developed  relative  risk  scoring 
systems  similar  to  the  HRS  to  determine  which  sites  on  their 
facilities  warranted  extensive  investigation.  The 
Department  Of  Energy  (DOE)  developed  the  Multimedia 
Environmental  Pollutant  Assessment  System  (MEPAS) .  "MEPAS 
is  used  to  prioritize  hazardous,  radioactive,  and  mixed- 
waste  sites,  based  on  their  potential  hazard  to  public 
health  (25 : iii )  .  " 

The  AF  developed  the  Hazard  Assessment  Risk  Model 
(HARM)  in  the  early  1980s  to  identify  IRP  sites  based  on  the 
initial,  limited  investigations  conducted  at  potential 
sites.  The  Defense  Priority  Model  (DPM)  evolved  from  the 
AF  HARM  with  extensive  revisions  and  peer  reviews  by 
numerous  experts  in  the  field  of  risk  assessment.  The 
DPM  was  developed  to  assist  AF  and  DOD  managers  in 
setting  priorities  for  funding  remedial  actions  at  IRP 
sites  (12 : 1-3) . 

The  DPM  is  used  after  extensive  Remedial  Investigations 
(RI)  have  been  completed  to  provide  a  numerical  score 
representing  the  relative  risk  to  human  health  and  the 
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environment.  The  DPM  calculates  relative  risk  by 
considering  the  type  and  amount  of  contaminant ( s )  along 
with  potential  transport  pathways  and  potential 
receptors  (12:1-3). 

Multi-Criteria  Evaluation 

If  there  were  only  one  criterion  or  factor  to  consider 
when  evaluating  alternatives  (progress  at  IRP  sites)  ranking 
the  alternatives  would  be  simple  (14:165).  However,  as  in 
the  case  of  this  research,  there  is  often  more  than  one 
factor  to  consider  when  evaluating  various  alternatives  such 
as  risk  reduction  and  site  closure  status.  Each  factor  can 
affect  the  evaluation  of  alternatives  individually  and  may 
be  difficult  for  the  decision  maker  to  compare  with  the 
other  factors  (29:139).  Multi-Criteria  Decision  Models 
(MCDM)  are  used  to  evaluate  each  of  the  alternatives  with 
respect  to  the  others  using  a  pre-specif ied  decision  rule  or 
set  of  rules  (2:8) . 

.Several  MCDM  techniques  have  been  developed  over  the 
last  fifteen  years  for  a  wide  range  of  applications.  There 
are  three  major  types  of  MCDM  techniques:  outranking, 
utility  based,  and  distance  based  models  (9:132).  The 
various  MCDMs  differ  in  how  the  criteria  are  weighted  to 
represent  their  relative  importance  with  respect  to  other 
criteria  (29:184). 

No  one  type  or  technique  of  MCDM  is  necessarily  the 
best  for  a  given  application  or  decision  maker  (9:134).  The 
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validity  of  an  MCDM  should  be  judged  by  how  it  measures  a 
decision  maker's  preferences  with  sufficient  accuracy  and 
consistency  (9:134).  The  following  provides  a  brief 
discussion  and  examples  of  the  three  types  of  MCDMs . 

Outranking  Models.  The  outranking  method  measures  the 
level  of  content  or  discontent  for  each  alternative  by  a 
pairwise  comparison  of  the  alternatives  for  each  criterion. 
An  advantage  of  this  method  is  "its  ability  to  incorporate 
qualitative  data  into  the  analysis  without  explicit 
quantification  of  criteria  ratings"  (8:16).  For  example,  it 
could  be  stated  that  cleaning  up  a  heavily  contaminated  site 
is  better  than  closing  a  site  with  little  or  no 
contamination  or  threat  to  the  environment. 

The  level  of  content  for  an  alternative  is  a  weighted 
measure  of  the  number  of  criteria  in  which  the  alternative 
is  preferred  to  another  alternative.  The  level  of 
discontent  is  measured  by  developing  a  scale  to  compare  each 
criterion  between  the  "best"  and  the  "worst"  for  that 
criterion.  The  level  of  content  and  discontent  are 
expressed  as  a  percentage  (8:17).  The  ELECTEE  MCDM  is  a 
widely  used  example  of  an  outranking  model  with  various  case 
studies  discussed  in  the  literature  (9:139).  "The  idea  in 
ELECTEE  is  to  choose  those  systems  that  are  preferred  for 
most  of  the  criteria  and  do  not  cause  an  unacceptable  level 
of  discontent  for  any  one  criterion"  (8:16). 

Utility  Based  Models.  Utility  or  value  based 
techniques  are  used  to  model  the  decision  maker's  preference 
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for  considering  the  various  factors  (or  attributes)  when 

evaluating  alternatives  (9:136). 

Utility  is  defined  as  the  subjective  benefits  derived 
by  the  decision-ma)cer  from  the  achievement  of  the 
stated  goals  or  objectives.  The  motivating  factor  of 
multiattribute  utility  theory  is  that  decision  ma)cer' s 
utility  function  can  be  specified  numerically.  (8:19) 

The  weight  for  each  factor  is  assigned  to  represent  the 

user's  value  or  utility  of  the  information  provided  by  the 

factor.  Mathematical  restrictions  require  that  the  factors 

considered  and  the  utility  or  value  assigned  to  the  factors 

must  be  mutually  independent  of  each  other  (8:19) .  For 

example,  a  decision  maker' s  utility  for  closing  out  a  site 

may  be  twice  as  important  as  whether  any  threat  to  the 

environment  has  been  eliminated.  Therefore,  the  attribute 

representing  environmental  risk  would  be  given  a  weight  one 

half  times  the  weight  of  the  attribute  for  closing  a  site. 

Distance  Based  Models.  The  concept  of  distance  based 

techniques  is  to  find  the  most  satisfying  alternative.  The 

alternative  selected  minimizes  the  distance  between  the 

alternative  and  a  reference  set  of  criteria  values  (9:135). 

"Distances  are  used  as  a  proxy  measure  for  human 
preference.  Distances  show  the  degree  of  resemblance, 
similarity,  or  proximity  of  alternatives  with  respect 
to  individual  criteria."  (9:135-136) 

There  are  several  types  of  distance  based  models,  they 
differ  in  the  way  a  reference  point  is  selected  and  how  they 
relate  to  the  reference  point.  In  most  cases  the  reference 
point  represents  an  infeasible  or  idea  alternative  in  which 
the  alternatives  are  related  (9:136).  An  example  of  this 
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technique  is  compromise  programming  and  the  displaced  ideal 
model  where  the  alternatives  are  ranked  by  their  closeness 
to  an  ideal  solution  (2:321-325). 

Another  type  of  distance  based  technique  is  cooperative 
game  theory  where  alternatives  are  compared  to  a  minimum 
level  reference  point.  The  objective  of  cooperative  game 
theory  models  are  to  rank  alternatives  based  on  the  maximum 
distance  from  the  reference  point  (8:19). 


III.  Methodology 


This  study  investigates  use  of  risk  reduction  as  a 
criterion  to  assess  progress  of  the  IRP.  The  first 
objective  was  to  demonstrate  a  method  to  quantify  risk 
reduction  at  IRP  sites.  Once  the  reduction  in  risk  can  be 
quantified,  the  second  research  objective  was  to  combine 
risk  data  with  other  factors  currently  used  to  measure 
progress.  A  multi-criteria  evaluation  technique  was  used  to 
rank  the  accomplishments  of  actions  taken  at  IRP  sites  using 
risk  data  along  with  other  factors. 

The  following  provides  a  description  of  the  methodology 
that  was  used  to  accomplish  the  research  objectives.  Figure 
1  on  page  15  shows  a  flow  diagram  of  the  steps  that  were 
followed  to  carry  out  the  research  objectives  described 
below.  The  results  of  the  research  were  presented  to 
e::perts  in  the  field  to  determine  if  the  methods  developed 
to  assess  progress  provide  a  viable  management  tool.  The 
methodology  to  analyze  and  evaluate  the  results  of  the 
research  is  also  described  in  this  chapter. 

Measure  Reduction  in  Risk 

Remedial  actions  at  IRP  sites  typically  involve 
reducing  the  volume  of  the  contaminants  and/or  reducing  the 
potential  for  the  contaminants  to  be  transported.  As 
progress  is  made  towards  cleaning  up  a  site,  various  factors 
used  to  measure  risk  change.  A  risk  model  that  is  currently 
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Figur*  1.  Flow  Diagram  of  the  Research  Methodology 


15 


used  to  measure  risk  at  IRP  sites  was  selected  to  measure 
reduced  risk  as  site  conditions  are  improved.  The  model 
selected  should  reflect  a  change  in  environmental  risk  as 
inputs  for  contaminant  levels  and  transport  pathways  are 
changed  as  result  of  remedial  efforts. 

To  evaluate  the  model's  application  as  a  tool  to  assess 
reduced  risks,  the  following  investigative  objectives  were 
carried  out: 

1.  Identify  the  input  variables  to  the  model  that  can 
be  changed  or  affected  by  remedial  actions. 

2 .  Rescore  sample  sites  by  changing  the  variable 
input  parameters  to  reflect  environmental 
conditions  after  remedial  actions  are  taken. 

The  purpose  for  rescoring  the  sites  was  to  investigate 
the  model's  sensitivity  to  the  input  parameters  that  can  be 
affected  by  remedial  efforts.  This  effort  was  simply  to 
determine  if  a  change  in  environmental  risk  can  be  measured 
with  the  model.  The  original  scores  were  compared  to  the 
rescore  values  to  determine  if  the  DPM  can  predictably  show 
a  reduction  in  risk.  It  was  not  the  focus  of  this  research 
to  evaluate  or  validate  the  methods  used  by  the  model 
selected  to  calculate  risk  at  contaminated  sites. 

Once  a  risk  model  was  selected,  sample  IRP  sites  were 
selected  to  measure  the  potential  risk  reduction.  The 
sample  IRP  sites  were  chosen  from  sites  where  the 
environmental  risk  has  previously  been  assessed  using  the 
risk  assessment  model  selected.  Each  site  was  rescored  by 
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changing  the  variable  input  parameters  to  reflect 
environmental  conditions  after  remedial  actions  have  taken 
place.  Each  variable  parameter  will  be  changed  to  represent 
the  best  possible  environmental  condition  achievable  by 
remedial  actions.  Although  the  remedial  actions  actually 
chosen  for  IRP  sites  may  not  result  in  achieving  the  best 
possible  scenario  for  every  parameter,  the  DPM  rescore 
represents  the  maximum  amount  of  reduction  in  risk  possible 
for  a  site. 

Combine  Risk  Data  with  Other  Factors 

A  multi-criteria  evaluation  method  was  selected  to 
consider  risk  data  along  with  other  administrative  factors 
currently  used  to  measure  progress.  The  MCDM  selected  was 
used  to  rank  the  accomplishments  at  the  sample  sites  with 
the  risk  reduction  data  generated  by  rescoring  the  DPM.  As 
discussed  in  the  literature  review,  the  various  MCDMs  differ 
in  how  the  criteria  are  weighted  to  represent  their  relative 
importance  with  respect  to  other  criteria  (2:184).  No  one 
type  or  technique  of  MCDM  is  necessarily  the  best  for  a 
given  application  or  decision  maker  (9:134).  The  validity 
of  an  MCDM  selected  was  evaluated  by  how  it  emulates  expert 
IRP  managers'  preferences  for  ranking  the  accomplishment  of 
IRP  actions. 

The  input  of  risk  data  to  the  MCDM  consisted  of  three 
measures  related  to  environmental  risk.  First,  the  original 
risk  score  or  in  other  words  the  measure  of  risk  that 
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represents  the  environmental  conditions  at  the  site  prior  to 
any  remedial  actions.  Second,  the  measure  of  potential  for 
risk  reduction  at  a  site.  This  was  measured  by  taking  the 
difference  between  the  original  score  from  the  risk  model 
selected  and  the  rescore  results  from  the  first  research 
objective.  The  third  input  related  to  risk  was  the  amount 
of  potential  risk  reduction  achieved  at  a  site.  This 
allowed  a  comparison  of  sites  where  the  most  extensive 
remedial  action  for  a  site  was  not  carried  out  or  where  a 
remedial  action  at  a  site  may  be  only  partially  completed. 

The  risk  data  was  combined  with  the  administrative 
measure  of  progress  currently  used  by  the  AF  which  is 
whether  the  site  has  been  closed  out  (as  defined  by  HQ 
USAF/CEV  guidance) .  For  the  purpose  of  demonstrating  the 
results  to  expert  IRP  managers,  the  data  used  in  the  MCDM 
for  the  site  closure  status  and  the  third  risk  factor 
(potential  risk  reduction  achieved)  was  made  to  represent 
realistic  site  scenarios.  The  data  for  these  two  factors 
had  to  be  made  up  since  the  sample  sites  have  not  been 
cleaned  up  yet  and  the  actual  data  was  not  available.  The 
data  was  made  up  to  allow  the  research  to  demonstrate  and 
evaluate  the  methods  developed  over  the  full  range  of 
various  IRP  site  status  scenarios.  Expert  interviews  used  to 
analyze  the  results  of  this  research  were  also  used  to 
ensure  that  the  range  of  scenarios  created  with  the  data  was 
reasonable  and  applicable. 
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Analysis  and  Evaluation 


To  evaluate  the  results  of  the  research  persoiial 
interviews  with  experienced  IRP  managers  were  conducted. 

The  interviews  consisted  of  presenting  the  results  of  the 
research  to  experienced  managers  to  determine  if  the  methods 
developed  provide  a  tool  for  managing  the  IRP.  The  experts' 
opinions  concerning  the  application  of  the  research  to  their 
management  concerns  were  documented  and  reported  as  part  of 
the  analysis  and  evaluation  in  Chapter  VI. 

An  unstructured  interview  was  used  to  allow  an  open 
exchange  of  the  experts'  specific  concerns  related  to 
measuring  progress  of  the  IRP.  This  interview  method  was 
selected  to  allow  the  researcher  to  explore  the  applications 
of  the  research  for  specific  IRP  management  concerns 
(10:324).  To  evaluate  the  research  the  experts  were  asked 
if  the  methods  developed  provide  a  viable  or  more  meaningful 
assessment  of  progress  for  managing  the  IRP. 

A  disadvantage  of  this  method  of  interviewing  is  that 
the  responses  may  have  been  biased  by  the  researcher's 
interview  techniques  (10:328).  This  disadvantage  will  be 
minimized  by  allowing  the  experts  to  address  their  specific 
concerns  related  to  measuring  progress  of  the  IRP.  However, 
this  research  was  based  on  the  assumption  that  the  lack  of 
using  risk  reduction  data  as  criteria  for  making  decisions 
is  a  common  concern  among  IRP  managers.  This  assumption  was 
supported  by  the  interviews  of  IRP  managers,  review  of 
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current  periodical  literature,  and  the  researcher's 
experiences  as  an  IRP  manager. 

The  experts  interviewed  were  selected  to  represent 
various  levels  of  the  AF  environmental  management 
organizational  structure.  Experts  at  HQ  USAF/CEVR  were 
selected  to  evaluate  the  research  from  the  perspective  of 
managing  the  AF-wide  IRP.  Experts  from  two  MAJCOMs  were 
selected  to  evaluate  the  application  of  the  research  for 
their  specific  management  requirements.  The  experts  from 
the  MAJCOMs  were  chosen  based  on  their  expression  of  a  need 
for  a  better  or  more  meaningful  method  to  assess  progress 
and  their  willingness  to  participate. 

Expert  selection  was  also  based  on  the  following 
qualities : 

1.  Position  as  an  IRP  manager  responsible  for 
executing  the  program. 

2 .  Experience  and  knowledge  of  the  management 
issues  related  to  the  IRP. 

3.  Authority  to  represent  their  organization  in  the 
interview. 
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IV, 


luantif ication  of  Reduction  in  Risk 


The  DPM  is  used  by  the  AF  and  DOD  to  determine  the  IRP 
sites  that  pose  the  greatest  risk  to  human  health  and  the 
environment.  The  application  of  the  DPM  to  measure 
reduction  in  risk  after  remedial  actions  have  been  taken 
will  be  investigated.  The  DPM  was  selected  for  this 
research  because  the  AF  currently  uses  the  model  to  assess 
the  relative  risk  of  all  IRP  sites  where  funding  for 
remedial  action  has  been  requested.  The  senior  managers  of 
the  AF  IRP  have  accepted  the  DPM  as  an  appropriate  tool  to 
measure  relative  risk  at  IRP  sites  (15) .  The  following 
provides  a  detailed  description  of  the  DPM  and  describes  how 
the  model  is  used  to  measure  reduction  in  risk  at  sample 
IRP  sites. 

Defense  Priority  Model  (DPM) 

The  DPM  was  developed  to  assist  AF  and  DOD  managers  in 
setting  priorities  for  funding  remedial  actions  at  IRP 
sites.  The  DPM  provides  a  method  to  rank  the  potential 
threat  to  human  health  and  the  environment  posed  by 
individual  sites.  A  score  is  calculated  for  each  IRP  site 
which  represents  the  relative  environmental  risk  at  the  site 
(21:1).  The  following  is  a  brief  discussion  on  the  history 
of  the  development  of  the  DPM  and  an  explanation  of  the 
methodology  used  to  calculate  relative  risk  scores  for  IRP 
sites . 
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History .  The  basic  philosophy  and  methodology  used  by 
the  DPM  was  initially  developed  for  the  AF  Hazard  Assessment 
Risk  Model  (HARM)  in  1984.  The  HARM  was  developed  during 
the  early  stages  of  the  IRP  (site  identification  phase)  to 
evaluate  potential  sites  for  further  consideration  in  the 
cleanup  program.  An  arbitrary  HARM  score  was  chosen  to 
delineate  which  potential  sites  would  be  identified  as  IRP 
sites  and  move  to  the  subsequent  phases  of  the  program.  The 
original  HARM  was  reviewed  and  tested  extensively  which  led 
to  significant  modification  and  the  development  of 
HARM  II  (12:2). 

HARM  and  HARM  II  were  developed  for  the  AF  under  an 
interagency  agreement  with  DOE  at  the  Oak  Ridge  National 
Laboratory  (12:2).  The  algorithms  and  inputs  for  the  model 
were  adapted  to  computerized  Artificial  Intelligence  (AI ) 
application  software  packages  (22:5). 

In  1987,  The  Office  of  the  OSD(E)  proposed  to  use  the 
method  developed  for  the  HARM  II  to  establish  a  risk  based 
priority  ranking  of  all  sites  in  DOD .  HARM  II  was  renamed 
the  DPM  and  DOD  continued  the  development  of  the  model 
(12:4).  DOD  continued  the  development  of  the  DPM  under  a 
contract  with  privately  owned  consulting  firms  (13). 

DOD  formally  announced  their  intention  on  using  the 
model  to  establish  a  risk  based  priority  for  allocating 
funds  for  remedial  action  and  solicited  comments  on  the 
HARM  II  methodology.  Comments  were  received  from  the  EPA, 
several  states,  and  other  interested  parties.  A  formal 
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response  was  provided  by  DOD  and  the  comments  were 
incorporated  into  DPM.  The  first  version  (version  2.0)  of 
the  DPM  was  released  for  use  by  the  DOD  components  in  1989. 
Version  3.0  and  Version  FY92  have  subsequently  been  released 
to  incorporate  additional  comments  received  from  EPA  and  the 
states  during  their  review  of  the  earlier  versions.  The 
automation  of  the  model  has  evolved  to  more  powerful  and 
sophisticated  AI  packages  and  C  computer  language  (22:1-7). 

To  validate  the  DPM' s  methodology,  the  model  was 
submitted  to  the  National  Academy  of  Sciences  (NAS)  for 
review  and  comment.  An  interim  response  from  the  NAS  was 
published  in  June  of  1992.  The  AF  and  DOD  are  currently 
developing  a  response  to  the  criticisms  and  concerns 
expressed  by  the  NAS  in  the  interim  report  (13) . 

Methodology  and  Structure.  The  DPM  provides  a 
numerical  score  representative  of  relative  risk.  The 
relative  risk  score  is  a  function  of  three  factors:  the 
contaminant (s)  hazard,  the  transport  pathways,  and  the 
potential  receptors.  The  DPM  is  made  up  of  three  segments 
to  address  each  of  the  factors  above.  First  scores  are 
generated  within  each  segment.  Subscores  are  then 
calculated  for  eight  scenarios  of  transport  pathways  and 
potential  receptors  (21:1).  The  final  score  is  calculated 
by  weighing  and  combining  the  subscores  using  an  exponential 
algorithm  (22:2-9,2-10). 

The  methodologies  for  scoring  of  each  of  the  segments 
of  the  DPM  are  briefly  discussed  below.  The  segments  are 
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described  in  the  order  in  which  data  is  input  into  the 
model.  First  the  pathway  segment  is  scored.  Second, 
contaminant  hazard  scores  are  calculated  for  each  pathway. 
The  potential  receptor  segment  is  scored  last.  The 
methodology  used  to  combine  the  segments  is  discussed  next. 
The  final  section  provides  a  description  of  the  algorithm 
used  for  the  computation  of  the  final  score.  The  complete 
algorithm  for  calculating  a  DPM  score  is  provided  in 
Appendix  A. 

Scoring  Pathways.  The  pathways  segment  of  the  DPM 
ranks  the  potential  for  contaminants  to  migrate  from  the 
disposal  site.  There  are  four  contaminant  transport 
pathways  considered  by  the  model.  The  pathways  are  "surface 
water  via  over  land  flow  routes,  the  groundwater,  or  the  air 
through  volatilization  or  adhered  to  dust"  (21:7).  A 
separate  score  is  calculated  for  each  pathway  (21:7). 

To  calculate  each  pathway  score  there  are  three 
components  of  information  concerning  the  conditions  at  a 
site.  The  first  type  of  information  relates  to  the 
potential  for  contaminants  to  enter  the  pathway.  There  are 
various  input  parameters  for  each  of  the  pathways  that 
indicate  the  relative  potential  for  the  contaminant  to  enter 
the  respective  pathways  (21:A-1).  The  second  component  is 
the  "waste  containment  effectiveness  factor"  (21:A-1).  This 
factor  indicates  the  potential  for  contaminants  to  migrate 
once  they  are  in  the  transport  media  The  third  component 
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is  the  amount  of  contaminant (s)  or  the  "waste  quantity 
factor"  (21:a-l)  associated  with  the  site  (21:A-1  to  A-4) . 

The  DPM  uses  two  different  methodologies  for  scoring 
each  pathway  depending  on  whether  a  contaminant  release  has 
been  observed  (measured) .  If  a  contaminant  release  is 
observed  in  a  transport  medium  the  score  for  the  potential 
for  the  contaminant (s)  to  enter  the  respective  pathway  is 
100  (maximum  normalized  score) .  The  input  parameters  to 
score  the  potential  for  the  contaminant (s)  to  enter  the 
pathway  (the  first  component)  are  skipped.  The  scores  for 
the  second  and  third  components  are  summed  and  normalized  to 
provide  the  overall  score  for  the  pathway  (21:A-1  to  A-4). 

If  no  release  has  been  observed  then  the  input 
parameters  for  the  first  component  are  used  to  score  the 
potential  for  a  contaminant  to  enter  the  pathway.  The 
resulting  score  is  normalized  and  the  multiplied  by  the  sum 
of  the  score  for  the  second  and  third  components.  The  new 
result  is  normalized  to  provide  the  overall  score  for  the 
pathway  (21:A-1  to  A-4).  A  copy  of  the  algorithm  for 
scoring  the  pathways  segment  of  the  DPM  by  hand  is  located 
in  Appendix  A,  pages  69  to  72. 

Scoring  Contaminant  Hazard.  The  contaminant 
hazard  scoring  segment  of  the  DPM  ranks  the  human  health 
hazards  and  ecological  hazards  of  the  contaminant (s)  of 
concern  at  a  site.  The  model  calculates  the  eight  following 
separate  hazard/pathway  scores  (21:55): 
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1.  human  health  hazards  of  surface  water  contaminants, 

2.  ecological  hazards  of  surface  water  contaminants, 

3.  human  health  hazards  of  ground  water  contaminants, 

4.  ecological  hazards  of  ground  water  contaminants, 

5.  human  health  hazards  of  air/soil  volatile  contaminants, 

6.  ecological  hazards  of  air/soil  volatile  contaminants, 

7.  human  health  hazards  of  air/soil  dust  contaminants,  and 

8.  ecological  hazards  of  air/soil  dust  contaminants.  (21:55) 

The  DPM  uses  various  methodologies  for  calculating 
hazard/pathway  scores  depending  on  whether  a  contaminant  has 
been  observed  (measured)  for  the  respective  pathway.  If 
contamination  has  been  detected  the  concept  of  Average  Daily 
Inta)ce  (ADI)  is  used  for  scoring  human  health  hazards.  The 
highest  observed  concentration  for  each  contaminant  is  used 
to  calculate  daily  inta)ce  which  is  divided  by  the  benchmarlc 
ADI  for  the  contaminant.  A  quotient  is  calculated  for  each 
contaminant  and  summed  to  provide  a  score  for  the  surface 
water  and  air/soil  pathways.  For  the  groundwater  pathways 
the  quotients  are  divided  by  a  retardation  factors 
calculated  for  the  respective  contaminants  and  then  summed 
to  provide  the  human  heath  hazard  score  (21:55) . 

The  procedure  for  calculating  ecological  health 
hazards  are  the  similar.  The  highest  observed 
concentrations  are  divided  by  benchmarJc  concentrations  for 
ecological  receptors  for  the  respective  pathway  (21:55). 

For  calculating  the  total  human  health  and  ecological 
hazard  scores  for  the  groundwater  and  surface  water  pathways 
all  contaminants  )cnown  to  be  present  at  a  site  are 
considered.  The  DPM  users  manual  list  two  conditions  that 
defines  contaminants  that  are  known  to  be  present: 
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1.  it  is  a  principle  component  of  the  materials  that 
were  placed  or  spilled  on  the  site,  or 

2.  it  has  been  detected  in  a  chemical  analysis  of 
site  soils  at  a  level  that  represents  an  increase 
above  background.  (21:61,68) 

Human  health  hazard  scores  for  contaminants  where  no 
detectable  concentration  levels  are  observed  the 
calculations  are  based  on  the  maximum  ADI.  For  the 
ecological  hazard  scores  the  calculations  are  based  on  the 
benchmarks  for  toxicity  for  the  appropriate  ecological 
receptors  (21:55).  When  no  detectable  concentration  of 
contaminants  can  be  observed  for  the  air/soil  volatile  and 
the  air/soil  dust  pathways  the  contaminant  hazard  scores  are 
set  to  zero  (21:75,78). 

Scoring  Receptors.  The  receptors  scoring  segment 
ranks  the  potential  for  humans  and  ecological  resources  to 
be  exposed  to  contaminates  that  have  migrated  from  a  site. 
The  model  calculates  scores  for  six  types  of  receptors: 

1.  human  health  receptors  of  surface  water  contaminants, 

2.  ecological  receptors  of  surface  water  contaminants, 

3.  human  health  receptors  of  ground  water  contaminants, 

4.  ecological  receptors  of  ground  water  contaminants, 

5.  human  health  receptors  of  air/soil  contaminants  and, 

6.  ecological  receptors  of  air/soil  contaminants  (21:79). 

Receptors  scores  for  the  air/soil  contaminants  are  scored 
once  which  includes  both  the  air/soil  dust  and  air/soil 
volatile  pathways  (21:79).  Each  of  the  receptors 
scores  are  calculated  base  on  input  parameters  which  provide 
information  about  the  land  uses,  water  resources, 
demographics,  and  sensitive  environmental  conditions  around 
the  site  (21:A-9  to  A-11). 
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Combining  Segment  Scores.  The  scores  from  the 
pathway,  contaminant  hazard,  and  receptor  segments  are 
multiplied  to  generate  eight  subscores.  Each  segment  score 
is  normalized  and  equally  weighted  (22:2-6).  The  subscores 
are  calculated  using  the  algorithm  shown  in  Figure  2.  The 
human  health  and  ecological  subscores  for  the  air/soil  dust 
and  air/soil  volatile  are  compared  to  select  the  most 
conservative  scores  for  the  air/soil  pathway.  The  highest 
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Figur*  2.  Algorithm  to  Calculate  DPM  Scores  (22:2-') 
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scores  for  the  air/soil  pathways  are  used  to  calculate  the 
final  DPM  score  (22:2-1)  . 


Calculating  the  Final  Score.  Six  pathway-hazard- 
receptor  subscores  are  combined  for  the  final  score.  The 
three  human  health  receptor  subscores  are  weighted  five 
times  more  than  the  ecological  receptor  subscores.  These 
weighing  factors  reflect  the  general  indication  of 
concern  in  national  environmental  regulatory  policy 
regarding  the  relative  importance  of  human  health  versus 
ecological  risks  (22:2-9). 

The  root-mean-square  methodology  is  use  to  combine  the 
subscores  to  generate  a  final  relative  risk  score.  The 
algorithm  for  computing  the  final  score  is  shown  in 
Appendix  A  on  page  73.  The  root-mean-square  method  is  an 
exponential  algorithm.  The  algorithm  results  in  a  high 
score  when  a  score  for  a  single  pathway-hazard-receptor 
score  is  high.  If  additional  subscores  are  high  the  final 
score  increases  but  not  linearly.  This  methodology  increase 
the  importance  of  a  single  high  pathway-hazard-receptor 
subscore  on  the  final  risk  score  (21:2-9,2-10) . 

Measure  Change  in  Relative  Risk 

Remedial  actions  at  IRP  sites  typically  involve 
reducing  the  volume  of  the  contaminants  and/or  reducing  the 
potential  for  the  contaminants  to  be  transported.  As 
progress  is  made  towards  cleaning  up  a  site,  input 
parameters  to  the  DPM  related  to  the  volume  and  potential 


29 


transport  of  the  contaminants  will  change.  The  DPM  score 
should  reflect  a  change  in  relative  risk  as  inputs  for 
contaminant  levels  and  transport  pathways  are  changed. 

Forty-eight  AF  IRP  sites  that  were  scored  using  the 
latest  version  of  the  automated  DPM  (ADPM92)  were  selected 
as  sample  sites  for  this  study.  The  sites  were  scored  by 
the  IRP  project  managers  at  the  respective  bases  using  data 
representative  of  the  site  conditions  prior  to  any  remedial 
action.  These  sites  were  selected  because  the  inputs  to 
generate  the  scores  were  reviewed  by  HQ  USAF/CEVR  to  insure 
Quality  Control  (QC)  standards  have  been  met. 

HQ  USAF/CEVR  provided  the  computerized  ADPM92  data  files  for 
the  sample  sites  used  in  the  research  (20) . 

To  evaluate  the  DPM' s  application  as  tool  to  assess 
reduced  risks,  the  following  investigative  objectives  were 
carried  out: 

1.  Identify  the  input  variables  to  the  DPM  that  can 
be  changed  or  affected  by  remedial  actions. 

2.  Rescore  the  sample  sites  by  changing  the  variable 
input  parameters  to  reflect  environmental 
conditions  after  remedial  actions  are  taken. 

Identify  Inputs  That  Can  Be  Changed.  The  DPM  has 
approximately  100  input  parameters  to  calculate  a  risk  score 
for  an  IRP  site.  The  first  step  will  be  to  examine  the 
input  parameters  and  separate  them  into  two  categories.  One 
category  is  parameters  that  the  input  value  cannot  be 
affected  by  remedial  actions.  Examples  of  this  category 
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will  include  input  values  related  to  climate^  demographics, 
and  geology  at  a  site.  The  second  category  is  made  up  of 
the  parameters  that  can  be  affected  by  remedial  efforts  such 
as  the  amount  of  contaminants  or  the  activity  at  a  site. 

There  were  two  input  parameters  that  could  not  be 
obviously  placed  into  one  of  the  two  categories  above. 

The  two  parameters  are  the  flooding  potential  in  the  surface 
water  pathway  section  and  the  groundwater  use  parameters  in 
the  human  health  receptors  for  groundwater  pathways  section. 

For  scoring  the  flooding  potential  at  a  site  the  ADPM92 
users  manual  states  that: 

Flooding  potential  is  a  measure  of  the  potential  for 
contaminants  to  be  transported  by  flood  waters. 

Flooding  potential  is  measured  by  the  frequency 
(observed  or  estimated)  of  inundation  due  to  stream 
flooding,  coastal  flooding,  high  lake  levels,  or  other 
causes.  (21:23) 

The  user's  manual  and  the  automated  version  of  the  DPM 
suggest  this  input  parameter  should  strictly  be  related  to 
the  site's  location  in  a  flood  plain  921:23).  Since  the 
regional  flood  plains  likely  would  not  be  changed  by 
remedial  efforts  this  would  not  allow  changing  the  input  for 
the  flooding  potential.  However,  remedial  measures  could  be 
taken  to  eliminate  the  potential  for  contaminants  to  be 
transported  as  a  result  of  flooding.  For  this  research  the 
input  for  flooding  potential  for  the  sample  sites  rescored 
was  set  to  the  lowest  potential  for  contaminant  transport  as 
a  result  of  flooding. 
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The  groundwater  use  of  the  aquifer  parameter  is  to 

evaluate  the  level  of  susceptibility  of  humans  ingesting 
contaminated  groundwater  (21:90).  Remedial  measures  often 
include  providing  an  alternate  water  source  to  a  population 
that  is  using  groundwater  from  a  contaminated  aquifer. 
Although  the  intended  measure  (the  population  using  the 
groundwater)  would  change,  there  has  been  no  progress  made 
toward  improving  the  environmental  conditions  at  the  site. 
The  groundwater  use  parameter  was  not  selected  to  be  changed 
since  the  objective  is  to  measure  progress  based  on 
improvements  in  environmental  conditions . 

The  following  is  a  list  of  the  parameters  that  were 
identified  and  selected  as  the  input  variables  to  the  DPM 
that  could  be  changed  by  remedial  actions. 

1.  The  surface  erosion  potential  for  the  surface 
water  pathway. 

2.  Flooding  potential  for  the  surface  water  pathway. 

3.  The  site  activity  for  the  air/soil  dust  pathway. 

4.  The  waste  containment  effectiveness  factor  for 
each  of  the  four  pathways . 

5.  The  waste  quantity  factor  for  each  of  the  four 
pathways  (when  applicable) . 

The  ability  or  appropriateness  to  change  the  waste 
quantity  factor  is  different  for  various  types  of  sites. 

The  waste  quantity  factor  is  related  to  the  amount  of 
contaminant ( s )  that  is  estimated  to  be  present  for  landfill, 
surface  impoundment,  and  Underground  Storage  Tank  (UST) 
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sites.  This  value  can  be  changed  by  remedial  actions.  The 
other  case  is  for  spill  and  Fire  Training  Area  (FTA)  sites 
where  the  waste  quantity  factor  is  a  descriptive 
characteristic  of  the  site  that  should  not  be  changed. 

The  waste  quantity  factor  for  FTAs  is  the  length  of 
time  the  site  was  used  as  a  fire  training  facility.  This 
cannot  be  changed  by  remedial  actions.  The  waste  quantity 
factor  for  spill  sites  is  the  amount  of  contaminant (s) 
involved  in  the  spill.  Although  the  contaminants  can  be 
removed  by  remedial  actions,  the  amount  of  contaminant 
involved  in  the  spill  cannot  be  changed.  The  input  values 
for  the  waste  quantity  factors  for  the  FTA  and  spill  site 
were  not  identified  as  parameters  to  be  changed  for 
rescoring  the  sample  sites. 

Rescore  IRP  Sites.  The  purpose  for  rescoring  the  sites 
is  to  demonstrate  the  DPM' s  sensitivity  to  the  input 
parameters  that  can  be  affected  by  remedial  efforts.  Each 
site  was  rescored  by  changing  the  variable  input  parameters 
to  reflect  environmental  conditions  after  remedial  actions 
have  taken  place.  Each  variable  parameter  was  changed  to 
the  best  possible  environmental  condition  considered  by  the 
DPM  for  the  parameter.  The  DPM  data  and  the  results  of 
rescoring  the  48  sample  sites  are  provided  in  Table  1  on 
pages  34  and  35.  Although  the  remedial  actions  actually 
chosen  for  IRP  sites  may  not  result  in  achieving  the  best 
possible  scenario  for  every  parameter,  the  DPM  rescore 
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Table  1 

DPM  Rescore  Data 


Base 

Site  Type 

Original 

Score 

Resoote 

Kelly 

Landfill 

46.0 

5.6 

40.4 

Wright-Pat 

Landfill 

61.8 

6.2 

55.6 

Kelly 

Spin 

48.0 

23.1 

24.9 

Kelly 

FTA 

36.2 

19.4 

16.8 

Tinker 

UST 

44.2 

7.9 

36.3 

Tucson 

Spin 

10.6 

3.3 

7.3 

England 

UST 

19.3 

4.7 

14.6 

MacDIll 

Spin 

21.5 

5.7 

15.8 

England 

Sf^n 

29.4 

15.3 

14.1 

Mt.  Home 

FTA 

24.1 

14.4 

9.7 

Davis  Monthan 

Waste  RIes 

20.5 

3.4 

17.1 

Tinker 

Waste  RIes 

9.0 

1.6 

7.4 

Tinker 

Landfill 

27.2 

2.7 

24.5 

Tinker 

UST 

37.9 

5.8 

32.1 

Kelly 

Landfill 

41.7 

5.4 

36.3 

Kelly 

Larxifin 

26.9 

5.0 

21.9 

Keify 

Surface  hnpourxfment 

28.7 

4.9 

23.8 

Wright-Pat 

Landfill 

60.7 

6.5 

54.2 

Wright-Pat 

Landfill 

51.1 

6.4 

44.7 

Kelly 

Surface  Impoundment 

23.7 

4.2 

19.5 

Tinker 

UST 

20.6 

6.0 

14.6 

Kelly 

Spill 

25.9 

13.5 

12.4 

Tinker 

Landfin 

10.6 

1.6 

9.0 

Tinker 

Landfin 

30.1 

3.5 

26.6 

Vance 

Surface  Impoundment 

17.0 

3.1 

13.9 

Vance 

FTA 

30.6 

19.1 

11.5 

Vance 

UST 

38.6 

6.0 

32.6 

Vance 

UST 

38.5 

6.0 

32.5 

Vance 

UST 

18.1 

5.9 

12.2 

Vance 

UST 

31.1 

5.9 

25.2 

Vance 

Landfin 

9.5 

2.0 

7.5 

Columbus 

Spin 

41,7 

19.6 

22.1 

Lackland 

Spin 

23.1 

12.4 

10.7 

Sheppard 

Surface  Impoundment 

6.7 

1.4 

5.3 

Beale 

Landfin 

40.8 

5.2 

35.6 

Beale 

UST 

38.3 

5.7 

32.6 

Vandenberg 

FTA 

19.8 

11.3 

8.5 

Vandenberg 

Spin 

39.9 

21.1 

18.8 

Vandenberg 

Surface  ImpourKfment 

16.3 

4.0 

12.3 

Vandenberg 

Spin 

4.7 

1.8 

2.9 

Malmstrom 

UST 

5.6 

0.6 

5.0 

Charleston 

Landfin 

10.0 

1.4 

8.6 

Charleston 

Other 

5.2 

1.3 

3.9 

Table  1  (cent.) 
DPM  Rescore  Data 


Base 

Site  Type 

Original 

Score 

Resoore 

Andrews 

FTA 

29.1 

11.8 

17.3 

Andrews 

Laxffln 

28.3 

5.6 

22.7 

Arxlrews 

Spin 

23.1 

10.9 

12.2 

McQuIre 

Spin 

32.3 

11.1 

21.2 

AF  Academy 

Landfill 

47.8 

4.8 

43.0 

represents  the  maximum  amount  of  reduction  in  risk  possible 
for  a  site. 

In  order  to  demonstrated  that  the  DPM  can  used  to 
measure  risk  reduction,  the  risk  scores  should  be  reduced 
when  the  input  parameters  identified  in  the  previous  section 
are  changed.  It  would  be  expected  that  for  higher  original 
DPM  risk  scores  the  would  be  a  greater  risk  reduction 
potential.  A  statistical  analysis  of  relationship  between 
the  original  DPM  scores  and  the  data  generated  by  rescoring 
the  sample  sites  is  discussed  in  Chapter  VI. 

This  effort  is  simply  to  determine  if  a  change  in 
relative  risk  can  be  measured  using  the  DPM.  It  is  not  the 
focus  of  this  research  to  evaluate  the  methods  used  by  the 
DPM  to  calculate  relative  risk  at  contaminated  sites  or  to 
validate  the  results  of  the  model. 
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V, 


Multi-Criteria  Evaluation  of  Progress 


Human  reasoning  desires  to  get  as  close  as 

possible  to  an  ideal  situation  or  one  that  is  better  than 

its  alternatives  (14:164). 

Although  it  is  difficult  to  determine  the  exact 
performance  of  a  facility  [or  environmental]  system,  it 
may  be  easier  to  define  a  system's  performance  [or 
accomplishments]  as  "better"  than  another's.  The 
performance  therefore  is  relative,  imprecise,  and 
"fuzzy".  .  .  .  The  relative  distance  from  an 

imaginary  optimum  or  maximum  value  of  the  fuzzy  set  can 
be  understood  in  relative  terms  of  far,  farther,  or 
close  and  closer.  By  using  relative  descriptions  of 
distance,  it  is  possible  to  arrive  at  a  value  that  is 
"close"  to  the  optimum.  (14:164) 

Quantitative  and  qualitative  measures  of  progress  can  be 

expressed  in  relative  terms  for  the  various  factors  used  to 

assess  progress.  Relative  or  ordinal  measurements  only 

determines  that  one  variable  for  a  factor  used  to  measure 

progress  is  greater  than  or  less  than  another  variable 

(10:173).  The  Displaced  Ideal  Model  (DIM)  incorporates  the 

ability  to  take  relative  measurements  of  factors  that 

indicate  progress  and  relate  them  to  the  ideal  solution 

(14:163-164).  The  DIM  was  selected  to  rank  the 

accomplishments  of  the  IRP  using  risk  data  and 

administrative  data  such  as  the  site  closure  status. 


Displaced  Ideal  Model  (DIM) 

The  DIM  is  a  variant  of  a  distance  based  model  where 
the  alternatives  are  compared  to  a  idea  reference  point 
(14:179).  As  discussed  in  the  previous  section  the 
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variations  in  MCDM  techniques  depend  on  how  the  factors  or 
attributes  are  weighted.  The  DIM  uses  two  considerations  in 
assigning  weights  to  each  factor  used  to  evaluate  the 
alternatives : 

1.  the  amount  of  information  that  is  contributed  by 
a  factor,  and 

2.  the  decisions  makers  strategic  weighing  of  the 
factors  (14:165) . 

The  amount  of  information  provided  by  a  given  factor  is 

determined  by  Zeleny's  adaptation  of  entropy  (29:190). 

Entropy  is  defined  by  Webster  as: 

A  measure  of  the  randomness,  disorder,  or  chaos  in  a 
system  specified  by  the  relationship  of  S  =  klnP  +  c, 
where  S  is  the  value  of  the  measure  for  a  system  in  a 
given  state,  P  is  the  probability  of  occurrence  of  that 
state,  and  k  is  a  fixed  and  c  is  an  arbitrary  constant. 
(28:457) 

The  measure  of  entropy  for  a  given  factor  is  inversely 
proportional  to  the  amount  of  information  provided  by  the 
factor.  When  the  measure  of  entropy  is  large  the  amount  of 
information  provided  by  that  factor  is  small.  Conversely, 
when  the  entropy  for  a  factor  is  small  the  amount  of 
information  provided  is  large  (14:178-179).  "The  principles 
of  the  'displaced  ideal'  and  'entropy'  are  used  to  determine 
the  effect  of  the  spread  of  values  within  each  criteria 
across  all  of  the  criteria"  (14:179). 

The  values  for  the  weights  generated  using  entropy  to 
determine  the  dominate  factor  are  multiplied  by  the  weights 
provided  by  the  decision  maker  for  each  factor.  If  there  is 
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no  dominant  factor  the  user's  weighing  of  the  factors  has  a 
larger  influence  of  the  results  of  evaluating  the 
alternatives.  When  the  user  does  not  differentiate  between 
the  factors  (weights  them  all  the  same)  the  model  is 
strictly  dependant  on  the  principles  of  entropy  assign  the 
weights  (14:165) . 

Combining  the  weights  generated  by  the  model  with  the 
user's  weighing  of  the  factors  allows  the  user  to  adjust  the 
model  to  meet  various  strategic  applications.  It  also 
allows  the  results  of  the  model  to  be  validated  in  the  sense 
that  it  measures  the  user's  or  expert's  preferences  with 
sufficient  accuracy  (29:195-198). 

Relative  Rank  of  Progress  for  IRP  Sites 

The  DIM  is  based  on  Zeleny's  adaptation  of  entropy  to 
assess  the  amount  of  information  an  attribute  provides 
(29:187-197).  The  entropy  and  DIM  principles  are  used  to 
determine  if  certain  factors  are  dominant  in  the  amount  of 
information  they  provide  with  respect  to  other  factors.  The 
DIM  weights  the  factors  according  to  the  relative  amount  of 
information  a  factor  provides  in  relation  to  the  other 
factors.  The  users  weighing  of  the  factors  are  combined 
with  weights  generated  by  the  DIM.  When  there  is  a  lack  of 
a  dominating  factor  the  model  relies  more  heavily  on  the 
users  weighing  of  the  factors  (14-165) . 

The  entropy  process  and  the  DIM  algorithm  developed  by 
Zeleny  used  to  rank  progress  for  the  sample  sites  in  this 
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research  is  described  by  the  following  steps  (14:180-182; 
29:197) : 

1 .  Enter  each  alternative  or  set  of  data  describing 

an  IRP  site  as  line  items  or  rows  into  a 
table:  x^,  ^ 

Where:  i  =  number  of  sites  or  alternatives  or 
rows,  and 

j  =  number  of  factors  or  columns. 

2.  Find  the  optimum  value  for  each  factor:  max^ 

3.  Find  the  percentage  of  the  optimum  value  for  each 

factor  of  each  site:  d,,  .= — 

^  ^  max^ 

4 .  Sum  the  percentage  of  each  optimum  value  {D^) 
for  each  factor: 

5.  Find  the  relative  distance  of  each  value  j) 
from  the  optimum  value:  k^,  ^=\ l-d^, 

1.  Find  the  distance  factor  by  dividing  the 

percentage  of  the  optimum  value  by  the  summed 

•  ^  i  f  i 

total  for  the  respective  factor: 

8.  Find  the  "entropy"  for  each  factor  for  each 


site  j)  using  the  natural  log  function: 


9.  Find  the  entropy  for  earh  of  the  factors  (edi^) 

by  dividing  the  sum  of  the  entropy  for  each  site 


by  the  maximum  entropy  value 

®max 

Where:  e„,,=ln(n),  n  =  number  of  sites. 


10.  Calculate  the  weights  representing  the  amount  of 

information  provided  by  each  factor  (X,)  by  the 
following  equation:  [I'edi^] 

11.  The  final  weights  (Xi)are  calculated  by  the 

1 

following  equation:  ; - 

Where:  w^=  the  weight  provided  by  the  user 


for  each  factor. 


12.  The  final  weights  (Xi^)  are  then  multiplied  by  the 
relative  distance  values  j)  for  the  respective 
factors  for  each  site:  K^,^=Xiyk.,^ 

13.  The  relative  rank  order  is  then  calculated  by 
summing  the  values  of  K  for  each  site: 

rank^Y^j  ^i’  i 


The  output  will  be  a  number  between  zero  and  one  for 
each  site.  The  output  represents  the  closeness  of  each  site 
to  the  optimum  or  idea  alternative.  If  each  of  the  factors 
considered  for  a  site  was  the  optimum  value  for  the 
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respective  factors  the  output  would  be  zero  or  an  optimum 
solution.  Likewise  the  higher  the  output,  the  further  the 
site  is  from  the  optimum  situation.  This  provides  a 
relative  ranking  of  the  sites  according  to  significance  of 
accomplishments  as  a  result  of  the  actions  taken. 

The  DIM  was  used  to  combine  the  risk  data  generated  by 
the  DPM  with  the  measure  of  progress  currently  used  by  the 
AF  which  is  whether  the  site  has  been  closed  out.  Input  of 
risk  data  consisted  of  three  measures  related  to  risk:  the 
original  DPM  score,  the  measure  of  potential  for  risk 
reduction,  and  the  amount  risk  reduction  achieved  at  a  site. 
The  results  of  using  these  criteria  and  the  DIM  to  rank  the 
progress  for  the  sample  sites  is  presented  in  Chapter  VI. 
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VI.  Analysis  and  Evaluation  of  Results 


Analysis  of  PPM  Rescore  Data 

In  order  to  demonstrated  that  the  DPM  can  used  to 
measure  risk  reduction,  the  risk  scores  should  be  reduced 
when  the  input  parameters  identified  in  Chapter  IV  are 
changed.  Each  site  was  rescored  by  changing  the  variable 
input  parameters  to  determine  the  maximum  amount  of  risk 
reduction  possible  for  each  sample  site.  It  would  be 
expected  that  for  higher  original  DPM  risk  scores  there 
would  be  a  greater  risk  reduction  potential. 

To  determine  if  there  is  any  relationship  between 
original  DPM  scores  and  the  results  of  rescoring  the  sites  a 
regression  analysis  was  performed.  As  the  original  DPM 
scores  increase  the  amount  of  potential  risk  reduction  by 
remedial  actions  also  increases.  This  was  determined  by 
statistical  evaluation  of  the  relationship  between  the 
original  site  scores  and  the  amount  of  potential  change  in 
the  DPM  score. 

There  is  a  nondeterministic  linear  relationship  that 
can  be  described  by  the  equation  Y  =  A(X)  +  B.  The 
independent  variable  (X)  is  the  original  site  score  and  the 
dependant  variable  (Y)  is  the  difference  between  the 
original  score  and  the  rescore  results  for  a  site.  A  plot 
and  regression  analysis  of  the  data  and  for  all  of  the 
sample  sites  is  shown  in  Figure  3.  Statistical  tests 
(student  t-test  and  F-test)  can  be  used  to  determine  the 
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Change  In  DPM  Sooraa  (Y) 


Original  Site  Sooraa  (X) 

Y  -  O.026O(X)  -  2.4230 
Correlation  Coefflolent :  0.0206 
Standard  Error  :  6.2302 


Figure  3.  DPM  Rescore  Data  for  All  Sites 

significance  of  the  correlation  of  the  two  variables 
determined  by  the  regression  analysis  (1) .  In  all  cases  the 
correlation  of  the  two  variables  exceeded  the  95  percent 
confidence  interval. 

Although  there  was  a  reasonably  good  correlation 
between  the  original  scores  and  the  rescores  for  all  of  the 


sites  considered  together,  a  more  precise  correlation  can  be 
shown  by  evaluating  the  data  by  site  type.  As  discussed  in 
the  previous  section,  the  manner  in  which  the  waste  quantity 
parameter  of  the  DPM  was  input  was  different  for  various 
types  of  sites.  This  had  an  effect  on  the  amount  DPM  scores 
could  be  reduced  as  the  original  scores  increased. 

When  the  waste  quantity  factor  is  changed  to  zero  to 
rescore  the  sites  (for  the  landfills,  surface  impoundments, 
and  USTs)  the  DPM  rescore  values  approach  a  constant  value. 
For  the  FTAs  and  Spill  sites  where  the  waste  quantity  input 
value  is  not  changed,  the  rate  of  increase  for  potential 
risk  reduction  as  the  original  DPM  score  increases  is  lower. 
Although  there  are  different  relationships  there  is  a  good 
correlation  between  original  scores  and  the  potential  risk 
reduction  for  each  of  the  site  types. 

The  difference  in  the  relationships  for  various  types 
of  sites  is  evident  from  the  slope  of  the  plot  of  the 
original  scores  versus  the  rescore  values  by  site  type 
(shown  in  Figure  4) .  This  difference  is  also  evident  from  a 
regression  analysis  of  the  data  for  the  various  site  types. 
The  following  provides  a  brief  description  of  the  analysis 
of  the  DPM  rescore  data  by  site  type. 

Landfill  Sites.  Fourteen  landfill  sites  were 
included  in  the  48  sample  sites.  The  landfill  sites  had  the 
largest  range  of  original  scores,  from  9.5  to  61.8,  and  the 
smallest  range  of  rescore  values,  from  1.4  to  6.5.  The  plot 
of  the  original  scores  versus  the  rescore  values  (Figure  4) 
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Reacore  Values  (Y) 
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Figure  4.  Original  0PM  Scores  versus  Rescores 


shows  that  there  is  the  largest  potential  change  in  DPM 
scores  as  the  original  scores  increase.  The  DPM  rescore 
data  for  the  landfill  sites  is  shown  in  Figure  5.  The 
regression  analysis  of  the  data  for  the  landfills  show  the 
highest  level  of  correlation  between  the  original  scores  and 
the  rescores.  The  rescore  values  for  all  landfill  sites 
approach  a  constant  score  of  approximately  six. 


Change  In  DPM  Scorea  (Y) 


Original  Site  Sooree  (X) 


Y  -  0.0007{X)  -  0.8667 
Correlation  Coefficient :  0.99B9 
Standard  Error  ;  0.6660 

Figure  5.  DPM  Rescore  Data  for  Landfill  Sites 


46 


UST  Sites.  Ten  of  the  48  sample  sites  rescored 
were  UST  sites.  There  original  DPM  scores  ranged  from  5.6 
to  44.2  and  the  rescore  values  ranged  from  0.6  to  7.9.  The 
rescore  values  for  the  UST  site  appears  to  approach  a 
constant  value  of  approximately  six.  A  regression  analysis 
of  the  correlation  between  the  original  scores  and  the 
potential  risk  reduction  is  shown  in  Figure  6.  The  high 
correlation  coefficient  value  and  low  standard  error 


Change  In  DPM  Scores  (Y) 


Y  -  0,8684(X)  -  1.1932 
Correlation  Coefflolent ;  0.9966 
Standard  Error  :  1.2708 


Figure  6.  DPM  Rescore  Data  for  UST  Sites 
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indicate  that  there  is  good  correlation  between  the  original 
site  score  and  the  amount  of  potential  risk  reduction. 

Surface  Impoundment  Sites.  There  were  five 
surface  impoundment  sites  that  range  from  6.7  to  28.7  for 
original  DPM  scores.  The  rescore  values  range  from  1.4 
to  4.9.  The  regression  analysis  that  describes  the 
relationship  between  the  DPM  scores  and  the  amount  of 
potential  risk  reduction  for  the  surface  impoundment  sites 
is  provided  in  Figure  7. 


Chang*  In  DPM  Scoraa  (Y) 


Y  -  0.1716(X)  -  0.2916 
Correlation  Coefflolent ;  0.9696 
Standard  Error  :  0.6134 


Figura  7.  DPM  Rescore  Data  for  Surface  Impoundment  Sites 
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Spill  Sites.  Eleven  of  the  48  sample  sites 
rescored  were  spilx  sites.  There  original  DPM  scores  ranged 
from  4.7  to  48.0  and  the  rescore  values  ranged  from  2 . 9  to 
24.9.  The  change  in  the  amount  the  risk  can  be  reduced  as 
the  original  DPM  score  increases  is  smaller  for  spill  sites 
since  the  waste  quantity  factor  was  not  be  reduced.  An 
analysis  of  the  correlation  between  the  original  scores  and 
the  potential  risk  reduction  is  shown  in  Figure  8. 


Change  In  DPM  Sooree  (Y) 


0  6  10  ie  to  to  eo  to  40  4S  oo 

Original  Site  Sooree  (X) 

Y  -  0.6003(X)  ♦  1.0176 
Correlation  Coefficient ;  0.9016 
Standard  Error  :  2.1065 

Figure  8.  DPM  Rescore  Data  for  Spill  Sites 
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FTA  Sites.  There  were  five  FTA  sites  that  range 
from  19.8  to  36,2  for  original  DPM  scores.  The  rescore 
values  range  from  8.5  to  16.8.  The  regression  analysis  that 
describes  the  relationship  between  the  DPM  scores  and  the 
amount  of  potential  risk  reduction  for  the  FTA  sites  is 
provided  in  Figure  9.  The  correlation  between  the  original 
score  and  rescore  values  for  the  FTAs  also  exceeds  the  95 
percent  confidence  interval. 


Change  In  DPM  Scores  (Y) 


Y  -  0.4Q84(X)  -  0.2812 
Correlation  Coefficient :  0.9382 
Standard  Error  :  2.4680 


Figure  9.  DPM  Rescore  Data  for  FTA  Sites 


Evaluation  of  the  Methods  Developed  to  Assess  Progress 


To  evaluate  the  methods  developed  to  compare  the 
accomplishments  of  actions  taken  at  IRP  sites,  a  sample  test 
case  was  used  to  demonstrate  the  results  to  expert  IRP 
managers.  Personal  interviews  with  the  IRP  managers  were 
used  to  compare  the  results  of  the  methods  developed  with 
their  views  on  how  progress  should  be  measured.  A  list  of 
the  IRP  managers  that  participated  in  the  sample  test  case 
is  provided  in  Appendix  B  on  page  74. 

Twenty  sites  from  the  48  sample  sites  that  were 
rescored  were  selected  to  demonstrate  the  DIM  ranking  of  the 
sites.  The  test  case  developed  to  demonstrate  the  DIM 
consisted  of  five  sites  from  each  of  four  different  types  of 
sites.  The  sites  were  selected  so  that  the  widest  range  of 
site  status  scenarios  could  be  represented.  This  was  done 
by  selecting  two  high  scores,  two  low  scores  and  one  mid 
range  score  from  each  of  four  site  types.  The  data  used  for 
the  potential  risk  reduction  achieved  and  the  site  closure 
factors  were  made  so  that  a  wide  variety  of  site  status 
scenarios  would  be  represented.  The  sample  site  data  used 
to  demonstrate  the  DIM  along  with  a  description  of  the  test 
case  data  is  provided  in  Appendix  C  on  page  75. 

Experts^  Ranking  of  Progress.  The  experts  were  asked 
to  rank  the  sample  sites  according  to  their  evaluation  of 
progress  using  the  sample  data  provided.  There  is  a 
reasonably  good  agreement  among  the  experts'  on  ranking  the 
sites  where  the  most  and  least  progress  has  been  made. 
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Table  2  on  page  53  shows  how  the  sites  are  ranked  by  each  of 
the  experts.  The  rank  order  for  the  sites  in  Table  2  is 
listed  in  the  order  of  the  total  ranking  which  is  the 
combined  scores  from  all  three  experts. 

When  each  of  the  managers  were  interviewed  they 
indicated  the  most  significant  criterion  they  used  for 
ranking  the  sites  was  the  amount  of  risk  reduced  at  a  site. 
Two  of  the  IRP  managers,  Mr.  Kneeling  and  Mr.  Ratliff  used 
the  amount  of  risk  reduced  as  the  primary  criterion  to  rank 
the  sites.  However,  they  used  two  different  methods  to 
consider  the  risk  reduction.  Mr.  Ratliff  ranked  the  sites 
using  the  change  in  risk  achieved  factor.  When  two  sites 
were  equal,  as  in  the  case  of  sites  P  and  Q  or  sites  K  and 
S,  he  then  used  the  site  closed  factor  to  rank  one  site  over 
the  other  (23) . 

Mr.  Kneeling  ranked  the  sites  by  assigning  each  site  to 
one  of  four  categories  representing  the  level  of  threat 
using  the  original  DPM  score.  DPM  scores  over  40  are 
considered  a  high  threat,  scores  between  20  and  40  are  a 
moderate  threat,  scores  between  10  and  20  are  a  low  threat 
and  scores  below  10  are  considered  no  threat.  He  then 
looked  at  which  category  the  sites  would  be  in  as  a  result 
of  the  actions  taken.  The  sites  which  moved  from  the  high 
threat  to  the  no  threat  categories  were  ranked  the  highest. 
The  sites  where  there  was  a  less  significant  or  no  change 
from  one  category  to  another  were  ranked  lower  accordingly. 
When  there  was  no  change  from  one  category  to  another 
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TabI*  2 

Reaulta  of  tho  Exports'  Ranking  of  Progroaa 
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Mr.  Kneeling  considered  the  actions  at  sites  with  low  or  no 
threat  to  be  the  least  significant.  Mr.  Kneeling  did  not 
consider  the  site  closed  criterion  at  all  to  rank  the 
progress  for  the  sites  (15)  . 

Ms.  Geil  used  a  qualitative  assessment  of  the  data 
provided.  She  thought  that  there  is  no  way  to  consider 
progress  strictly  quantitatively,  there  must  be  some 
qualitative  assessment  by  expert  managers.  She  ranked  the 
site  using  three  primary  considerations:  the  amount  of  risk 
reduced,  the  residual  risk  at  a  site,  and  whether  the  site 
has  been  closed.  Instead  of  using  HQ  USAF/CEV  guidance  for 
determining  a  'yes'  or  'no'  for  site  closed  she  suggested 
there  needs  to  be  a  qualitative  assessment  of  the  regulatory 
and  community  relationships  related  to  IRP  actions  (11). 

Displaced  Ideal  Model  (DIM)  Results.  To  demonstrate 
the  DIM,  the  users'  input  for  weighing  each  of  the  factors 
were  set  to  be  equal  (a  weight  of  .25  for  each  of  the  four 
factors  considered) .  This  allowed  the  weighing  of  each 
factor  to  be  assigned  according  to  the  amount  of  information 
provided  by  each  factor  as  determined  by  the  entropy 
function.  The  weights  assigned  to  each  factor  using  the 
entropy  function  are  as  follows:  original  DPM  score  10.85, 
change  in  risk  possible  17.39,  change  in  risk  achieved 
32.38,  and  site  closed  39.17. 

Relying  on  the  entropy  function  alone  to  assign  weights 
for  the  factors  did  not  result  in  an  accurate  simulation  of 
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the  experts'  interviewed  ranking  of  the  sites.  The  results 
of  using  the  DIM  to  rank  the  sample  sites  are  shown  in 
Table  3  on  page  56.  Although  the  DIM  accurately  simulated 
the  experts'  ranking  for  the  top  three  sites,  the  overall 
accuracy  was  poor.  This  was  due  to  the  relatively  high 
weight  given  to  the  site  closed  criterion  and  low  weight 
given  to  the  change  in  risk  when  compared  to  expert's 
consideration  of  these  two  factors. 

The  users  input  for  weighing  of  each  of  the  factors  can 
be  adjusted  so  that  the  model  can  be  used  for  various  users 
management  strategies.  By  trial  and  error  the  user  input 
weights  were  adjusted  so  that  the  DIM  better  simulated  the 
results  of  the  experts'  ranking  of  the  sites.  The  user 
input  weights  assigned  to  each  factor  are  as  follows: 
original  DPM  score  20,  change  in  risk  possible  10,  change  in 
risk  achieved  65,  and  site  closed  5.  The  results  of  using 
the  DIM  with  the  adjusted  user  weights  is  shown  in  Table  4. 

By  combining  the  users'  input  weight  described 
above  with  the  weights  generated  by  the  entropy  function  the 
final  weights  are  8.02,  6.43,  78.3,  and  7.24  respectively. 
These  weights  more  closely  represent  the  experts' 
consideration  of  the  factors  when  they  were  asked  how  they 
went  about  ranking  the  sites.  The  results  show  that  the  DIM 
can  be  used  to  simulate  the  experts'  ranking  of  the  site 
with  reasonable  accuracy. 
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Table  3  Table  4 

DIM  Ranking  of  Progress  Adjusted  DIM  Ranking  of  Progress 
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Site  I  13  20  Site  J 


VII.  Conclusion 


Overview 

Measuring  the  number  or  percent  of  sites  closed  does 
not  reflect  what  has  been  done  to  improve  the  environmental 
conditions  at  IRP  sites  (3:144).  At  many  of  the  closed 
sites  no  remedial  actions  have  been  taken  at  all.  Currently 
the  AF  has  no  way  to  compare  the  accomplishments  at  the  most 
significant  cleanup  efforts  (at  the  worst  sites)  to  sites 
where  no  remedial  action  is  required.  To  properly  manage 
the  IRP  the  AF  needs  to  be  able  to  assess  progress  based  on 
tangible  environmental  improvements. 

An  important  part  of  evaluating  progress  toward 
protecting  human  health  and  the  environment  is  the 
assessment  of  risk  reduction  as  a  result  of  cleanup  actions. 
The  AF  currently  uses  the  DPM  to  assist  in  setting 
priorities  for  funding  remedial  actions  based  on  the 
relative  risk  at  IRP  sites.  The  DPM  provides  a  numerical 
score  representing  the  relative  potential  risk  based  on  the 
environmental  conditions  at  a  site. 

The  purpose  of  this  study  was  to  develop  a  method  to 
use  risk  reduction  as  a  criterion  to  assess  progress  for  the 
IRP.  Once  a  method  is  available  to  quantify  reduction  in 
risk,  the  risk  data  can  be  combined  with  other  factors  to 
rank  progress  using  various  multi-criteria  evaluation 
techniques . 
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The  following  section  sununarizes  findings  of  using  the  DPM 
to  measure  reduced  risk  and  using  the  DIM  to  rank  progress. 
The  last  two  sections  discuss  the  application  of  the 
research  and  recommendations  for  further  development  of 
methods  to  use  a  risk  based  approach  for  managing  the  IRP . 

Summary  of  Findings 

Using  the  DPM  to  Quantify  Risk  Reduction.  The  DPM 
calculates  relative  risk  by  considering  the  type  and  amount 
of  contaminant (s)  along  with  the  potential  transport 
pathways  and  the  potential  receptors.  As  progress  is  made 
toward  cleaning  up  a  site,  input  parameters  to  the  DPM 
related  to  the  volume  and  potential  transport  of  the 
contaminants  change.  Rescoring  the  DPM  to  represent 
conditions  after  remedial  actions  at  sample  sites  shows  a 
reduction  in  relative  risk.  Using  the  DPM  to  measure  risk 
before,  during,  and  after  remedial  actions  can  provide  an 
indicator  of  progress  based  on  improvements  in  environmental 
conditions  at  IRP  sites. 

Using  the  DIM  to  Rank  Progress.  By  comparing  the 
results  of  the  expert's  to  the  results  of  the  DIM  it  was 
determined  that  the  DIM  could  be  used  to  simulate  the 
expert's  ranking  of  progress.  The  DIM  needs  to  be  adjusted 
and  tested  to  ensure  that  it  accurately  reflects  the  users' 
specific  concerns  or  management  strategies  related  to 
measuring  progress  for  the  IRP. 
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Applications  for  Managing  the  IRP 


An  important  part  of  the  personal  interviews  was  to 
determine  if  there  is  an  application  for  methods  developed 
to  provide  a  more  meaningful  assessment  of  progress  to 
manage  the  IRP.  Each  of  the  managers  interviewed  believe 
that  a  risk  based  assessment  of  progress  is  more  appropriate 
than  using  administrative  milestones  such  as  the  number  of 
sites  closed.  Each  of  the  experts  indicated  that  there  are 
potential  applications  they  could  foresee  for  the  methods 
developed  in  this  research.  However,  all  of  the  experts 
feel  that  the  most  useful  application  would  involve  using 
risk  reduction  data  along  with  cost  to  rank  progress  or  set 
priorities  for  the  IRP. 

There  are  two  specific  potential  applications  for  using 
a  risk  based  assessment  of  progress  that  were  brought  up  in 
the  expert  interviews.  The  first  application  is  to  provide 
managers  a  method  to  compare  IRP  projects,  base  programs,  or 
MAJCOM  programs  based  on  their  accomplishments  toward 
improving  environmental  conditions  (15) . 

The  current  methods  of  evaluating  IRP  projects  or 
programs  give  credit  only  when  sites  can  quickly  be  closed. 
Typically  only  sites  where  no  or  very  insignificant 
improvements  of  environmental  conditions  have  been  made  have 
been  closed.  At  many  of  the  closed  sites  no  remedial 
actions  have  been  taken  at  all.  There  is  no  way  of 
differentiating  the  amount  of  significant  improvements  in 
environmental  conditions  as  a  result  of  the  actions  taken  at 
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a  site.  A  risk  based  assessment  provides  a  method  to  show 
progress  at  the  most  significant  cleanup  efforts  (at  the 
worst  sites)  that  take  many  years  to  complete  (18) .  By 
considering  the  cost  along  with  the  amount  of  risk  reduction 
the  managers  would  be  able  to  consider  the  "bang  for  the 
buck"  for  the  actions  taken  at  IRP  sites  (18) . 

The  second  potential  application  of  the  methods 
developed  was  for  prioritizing  potential  cleanup 
alternatives.  The  DPM  can  be  rescored  for  the  level  of 
various  potential  cleanup  alternatives.  Then  the  DPM  risk 
reduction  data  could  be  used  along  with  estimated  cost  to 
prioritize  and  select  remedial  alternatives  (15/18). 

Recommendations 

There  will  most  likely  continue  to  be  an  increasing 
level  of  interest  and  oversight  of  environmental  cleanup 
programs  such  as  the  IRP.  It  is  important  for  the  AF  to  be 
able  to  show  progress  toward  restoring  the  environment  and 
eliminating  the  threat  posed  by  the  worst  sites.  To  justify 
and  properly  manage  resources  for  the  IRP,  the  AF  needs  to 
employ  risk  based  methods  to  assess  progress. 

In  order  to  further  develop  and  implement  the  methods 
used  in  this  research,  test  cases  using  actual  data  from 
sites  where  cleanups  are  underway  should  be  investigated. 

The  biggest  benefit  of  using  data  representing  actual  site 
scenarios  will  be  to  allow  considering  cost  as  a  factor  to 
consider  with  the  risk  data.  Cost  or  "bang  for  the  buck" 
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would  be  weighted  very  heavily  for  the  potential  application 
to  measure  progress  or  select  remedial  action  alternatives 
(15; 18).  Cost  of  the  remedial  action  taken  was  not  used  as 
a  criterion  for  this  research  because  no  data  was  available 
for  the  sample  sites . 

A  comparison  of  cleanup  actions  at  the  worst  sites  to 
sites  where  little  or  no  action  was  required  to  close  the 
site  using  actual  site  status  data  should  be  investigated. 
This  will  require  rescoring  the  DPM  for  sites  where  cleanup 
has  taken  place  to  represent  the  actual  level  of  action 
taken.  This  could  include  sites  where  the  cleanup  has  not 
been  completed.  An  original  DPM  score  representing  the  risk 
at  sites  where  no  remedial  actions  where  required  to  close 
the  site  would  also  need  to  be  generated.  Typically  the  DPM 
has  only  been  scored  for  sites  when  there  is  a  request  for 
DERA  funding  to  take  remedial  actions.  The  actual  cost  of 
the  actions  taken  for  a  limited  number  of  sites  should  be 
available  to  use  in  the  investigation. 

Another  area  that  may  be  investigated  is  the 
application  of  using  the  DPM  to  select  remedial  alternatives 
at  a  site.  The  DPM  could  be  scored  to  represent  several 
various  site  conditions  that  would  be  a  result  of  the 
alternative  selected.  The  estimated  cost  of  each  of  the 
alternatives  along  with  likelihood  of  regulatory  and/or 
public  acceptance  of  the  action  could  be  considered  with  the 
risk  data. 
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SURFACE  WATER  PATHWAY 


Detected  Releases 


Score 

(circle  one)  Multiplier 


1.  Have  contaminants  been  detected  in  0  100 
surface  water?  If  yes,  assign  score 

of  100  and  proceed  to  item  [10].  If 
no,  assign  a  score  of  0  and  proceed 
to  item  [2]. 

Pathway  Characteristics 

2.  Distance  to  nearest  surface  water  0  12  3 

3.  Net  precipitation  0  12  3 

4.  Surface  erosion  potential  0  12  3 

5.  Rainfall  mtensity  0  12  3 

6.  Surface  hydraulic  conductivity  0  12  3 

7.  Flooding  potential  0  12  3 

8.  Sum  of  items  [2]  through  [7] 

9.  Normalized  score  (item  [8]  x 
100/78) 

10.  Waste  contaimnent  effectiveness 
factor 


1 


4 

1 

4 

4 

3 

10 


11.  Waste  quantity  factor 

12.  Final  pathway  score  for  surface 
water  (item  [9]  x  (item  [10]  + 
item  [ll])/2). 


Score 

Product 

(score  X  Maximum 

mult.)  Score 

100 


12 

3 

12 

12 

9 

30 

78 
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GROUND  WATER  PATHWAY 


Detected  Releases 

Score 
(cirde  one) 

Multiplier 

Score 
Product 
(score  X 
mult.) 

Maximum 

Score 

13.  Have  contaminants  been  detected  in 
groimd  water?  If  yes,  assign  score 
of  100  and  proceed  to  item  [20].  If 
no,  assign  score  of  0  and  proceed  to 
item  [14]. 

0  100 

1 

100 

Pathway  Characteristics 

14.  Distance  to  seasonal  high  groimd 
water  from  base  of  waste  or 
contaminated  zone  and  potential  for 
discrete  features  in  the  unsaturated 
zone  to  "short  circuit"  the  pathway 
to  the  water  table 

0  12  3 
456 

10 
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IS.  Hydraulic  conductivity  of  the 
unsaturated  zone 

0  123 

5 

15 

16.  Infiltration  potential 

0  123 

5 

1 

15 

17.  Geochemical  properties  of  the 
vadose  zone 

0  123 

5 

15 

18.  Sum  of  items  [14]  through  [17] 

105 

19.  Nomudized  score  (item  [18]  x 
100/105) 


20.  Wiste  conUinmeat  effectiveness 
factor 

21.  Waste  quantity  factor 

22.  Final  pathway  score  for  ground 
water  (item  [19]  x  (item  [20]  + 
item  [21])/2) 
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AK/SOIL  VOLATILES  PATHWAY 


Detected  Rdeases 


Score 

(cirde  one)  Multiplier 


23.  Have  volatile  contaminants  been  0  100 

detected  in  ambient  air  above 
background  levels?  If  yes,  assign 

score  of  100  and  proceed  to  item 
P2].  If  no,  assign  score  of  0  and 
proceed  to  item  [24]. 

Pathway  Characteristics 

24.  Have  volatile  contaminants  been  0  3 

detected  in  surface  soil?  If  yes, 

assign  a  score  of  3  and  proceed  to 
item  [25].  If  no,  assign  a  score  of 
0  to  items  [24]  and  P4],  and 


proceed  to  item  [35]. 

25.  Average  summer  soil  ten^rature  0  12  3 

26.  Net  precipitation  0  12  3 

27.  Wind  velocity  0  12  3 

28.  Soil  porosity  0  12  3 

29.  Sum  of  items  [24]  through  [28] 


30.  Normalized  score  (item  [29]  x 
100/60) 

31.  Adjusted  pathways  score.  If  item 
[23]  is  100,  enter  100.  If  item  [23] 
is  0  and  item  [24]  is  0,  enter  0.  If 
item  [24]  is  not  0,  enter  value  from 
item  PO]. 

32.  Waste  containment  effectiveness 
factor 


1 


12 


2 

2 

2 

2 


33.  Waste  quantity  factor 

34.  Final  pathway  score  for  air/soil 
volatiles  (item  Pl]  X  (item  P2]  + 
item  P3])/2) 


Score 

Product 

^core  X  Maximum 

mult.)  Score 

100 


36 


6 

6 

6 

6 

60 
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AK/SOa.  DUST  PATHWAY 


Detected  Releases 

Score 
(circle  one) 

Multiplier 

Score 
Product 
(score  X 
mult.) 

Maximum 

Score 

35.  Have  non-volatilecontaminants  been 
detected  in  ambient  air  above 
background  levels?  If  yes,  assign 
score  of  100  and  proceed  to  item 
[44].  If  no,  assign  score  of  0  and 
proceed  to  item  P6]. 

0  100 

1 

100 

Pathway  Characteristics 

36.  Have  non-volatilecontaminants  been 
detected  in  the  surface  soil?  If  yes, 
assign  a  score  of  3  and  proceed  to 
item  P?].  If  no,  assign  a  score  of 

0  to  items  P6]  and  [46],  and 
proceed  to  item  [47]. 

0  3 

12 

36 

37.  Net  precipitation 

0  123 

2 

6 

38.  Wind  velocity 

0  123 

2 

6 

39.  Days/year  >  0.25  mm  precipitation 

0  123 

2 

- j- 

6 

40.  Site  activity 

0  1  23 

2 

6 

41.  Sum  of  items  [36]  through  [40] 

_ 

60 

42.  Normalized  score  (item  [41]  X 
100/60) 

43.  Adjusted  pathways  score.  If  item 
PS]  is  100,  enter  100.  If  item  P5] 
is  0  and  item  P6]  is  0,  enter  0.  If 
item  P6]  is  not  0,  enter  value  from 
item  [42]. 

44.  Waste  containment  effectiveness 
factor 

— 

45.  Waste  quantity  factor 

46.  Final  pathway  score  for  air/soil  dust 
(item  [43]  x  (item  [44]  +  item 
[45])/2) 
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CONTAMINANT  HAZARD-SURFACE  WATER 

Score 

(drcie  one)  Result 

47.  Sum  of  Health  Hazard  Quotients 
(from  column  9  of  the  Surface 
Water  Hazard  Worksheet  for 
detected  releases). 

48.  Human  Health  Hazard  Score  (Table 
G-1). 

0  123 

45  6 

49.  Final  Health  Hazard  Score  for 
surface  water  pathway  (item  [48]  x 
100/6). 

SO.  Sum  of  Ecological  Hazard  Quotients 
(enter  the  larger  of  the  sum  of 
column  10  of  the  Surface  Water 
Hazard  Worksheet  for  detected 
releases). 

SI.  Ecological  Hazard  Score  (Table 
G-2). 

0  123 

4S  6 

S2.  Final  Ecological  Hazard  Score  for 
surface  water  pathway  (item  [51]  x 
100/6). 

— 

S3.  Maximum  Health  Hazard  Score 
(from  column  2  of  the  Surface 
Water  Hazard  Worksheet  for  non- 
detected  releases). 

0123456789 

Contaminant: 

S4.  Final  Health  Hazard  Score  for 
surface  water  pathway  (item  [.'  '  j  x 
100/9). 

SS.  Maximum  Ecological  Hazard  Score 
(from  column  3  of  the  Surface 
Water  Hazard  Woiksheet  for  non- 
detected  releases). 

0123456 

Contaminant- 

S6.  Final  Ecological  Hazard  Score  for 
surface  water  pathway  (item  [55]  x 
100/6). 
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CONTAMINANT  HAZARD-GROUND  WATER 

Score 

(circle  one)  Result 

57.  Sum  of  Health  Hazard  Quotients 
(from  column  9  of  Ground  Water 
Hazard  Worksheet  for  detected 
releases). 

- 

58.  Human  Health  Hazard  Score  (Table 
G-1). 

0  123 

456 

59.  Final  Health  Hazard  Score  for 
ground  water  pathway  (item  [58]  x 
100/6). 

60.  Sum  of  Ecological  Hazard  Quotients 
(column  10  Ground  Water  Hazard 
Worksheet  for  detected  releases). 

61.  Ecological  Hazard  Score  (Table 
G-2). 

0  123 

45  6 

62.  Final  Ecological  Hazard  Score  for 
ground  water  pathway  (item  [61]  x 
100/6). 

63.  Maximum  Health  Hazard  Score 
(from  column  11  of  the  Ground 
Water  Hazard  Worksheet  for  non- 
detected  releases). 

0123456789 

Cnntaininant: 

64.  Final  Health  Hazard  Score  for 
ground  water  pathway  (item  [63]  x 
100/9). 

65.  Maximum  Ecological  Hazard  Score 
(from  column  12  of  the  Ground 
Water  Hazard  Worksheet  for  non- 
detected  releases). 

0  1  23  456 

Contaminant; 

66.  Final  Ecological  Hazard  Score  for 
ground  water  pathway  (item  [65]  x 
100/6). 
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CONTAMINANT  HAZARD-AIR/SOIL  VOLATILES 


Score 
(circle  one) 


67. 

Sum  of  Health  Hazard  Quotients 
(from  column  7  of  the  Air/Soil 
Volatile  Hazard  Woricsheet). 

68. 

Human  Health  Hazard  Score  (Table 
G-1). 

0  12  3 
456 

69. 

Final  Health  Hazard  Score  for 
air/soil  volatile  pathway  (item  [68] 

X  100/6). 

70. 

Sum  of  Terrestrial  Hazard  Quotients 
(from  column  8  of  the  Air/Soil 
Volatile  Hazard  Worksheet). 

71. 

Ecological  Hazard  Score  (Table 
G-2). 

0  12  3 
456 

72. 

Final  Ecological  Hazard  Score  for 
air/soil  volatile  pathway  (item  [71] 

X  100/6). 

Result 
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CONTAMINANT  HAZARD-AIR/SOIL  DUST 

Score 
(circle  one) 


77.  Sum  of  Health  Hazard  Quotients 
(from  column  9  of  the  Air/Soil  Dust 
Hazard  Woricsheet). 

78.  Human  Health  Hazard  Score  (Table  0  12  3 

G-1).  4  5  6 

79.  Final  Health  Hazard  Score  for 
air/soil  dust  pathway  (item  [78]  x 
100/6). 

80.  Sum  of  Terrestrial  Hazard  Quotients 
(from  column  10  of  the  Air/Soil 
Dust  Hazard  Worksheet). 

81.  Ecological  Hazard  Score  (Table  0  12  3 

G-2).  4  5  6 

82.  Final  Ecological  Hazard  Score  for 
air/soil  dust  pathway  (item  [81]  X 
100/6). 


Result 
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HUMAN  HEALTH  RECEFTORS-SURFACE  WATER 

PATHWAY 

Score 

(circle  one) 

Multiplier 

Score 
Product 
(score  X 
mult.) 

Maximum 

Score 

87.  Population  that  obtains  drinking 
water  from  potentially  affected 
surface  water  bodyfies)  downstream 

0  123 

3 

9 

88.  Water  use  of  nearest  surface  water 
body(ies) 

0  123 

3 

9 

89.  Population  within  ‘A  mi  (806  m)  of 
the  site 

0  123 

1 

3 

90.  Distance  to  the  nearest  installation 
boundary 

0  123 

1 

3 

91.  Land  use  and/or  zoning  within  2 
miles  (3.2  km)  of  the  site 

0  123 

1 

3 

92.  Sum  of  items  [87]  through  [91] 

27 

93.  Final  Human  Health  Receptors 
score  for  surface  water  pathways 
(item  [92]  x  100/27) 

ECOLOGICAL  RECEPTORS-SURFACE  WATER  PATHWAYS 

94.  Importance/sensitivity  of  biota/ 
habitats  in  potentially  affected 
surface  water  bodies  nearest  the  site 

0  123 

5 

15 

95.  Presence  of  'critical  environments* 
within  1 .5  miles  (2.4  km)  of  the  site 

0  3 

1 

— 

3 

96.  Sum  of  items  [94]  and  [95] 

97.  Final  Ecological  Rec^tors  score  for 
surface  water  pathways  (item  [S16] 

X  100/18) 

18 
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HUMAN  HEALTH  RECEFTORS-GROUND  WATER  PATHWAY 

Score 

(circle  one) 

Multiplier 

Score 
Plnoduct 
(score  X 
mult.) 

Maximum 

Score 

98. 

Estimated  mean  ground  water 
travel  time  from  waste  location  to 
nearest  downgradient  water 
supply  well(s) 

0  123 

9 

27 

99. 

Estimated  mean  ground  water 
travel  time  from  current  waste 
location  to  any  downgradient 
surface  water  body  that  supplies 
water  for  domestic  use  or  for 
food  chain  agriculture 

0  123 

5 

15 

100. 

Ground  water  use  of  the  upper¬ 
most  aquifer 

0  123 

4 

— 

12 

101. 

Population  potentially  at  risk  from 
ground  water  contamiiuition 

0  3  6  9  12 

18  24  27 

36 

1 

— 

36 

102. 

Population  within  'A  mi  (806  m) 
of  the  site 

0  123 

1 

— 

3 

103. 

Distance  to  the  nearest  installation 
boundary 

0  123 

1 

— 

3 

104. 

Sum  of  items  [98]  through  [103] 

96 

105. 

Final  Human  Health  Receptors 
score  for  ground  water  pathways 
(item  [104]  x  100/96) 

ECOLOGICAL  RECEPTORS-GROUND  WATER  PATHWAYS 

106. 

Estimated  mean  ground  water 
travel  time  from  waste  location  to 
any  downgradient  habitat  or 
natural  areas 

0  123 

3 

— 

9 

107. 

Inqwrtance/sensitivity  of  down- 
gradimt  biota/habitats  that  are 
confirmed  or  suspected  ground 
water  discharge  points 

0  123 

3 

9 
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HUMAN  HEALTH  RECEPTORS-GROUND  WATER  PATHWAY  (concluded) 

Score 

Product 

Score 

(score  X 

Maximum 

(circle  one) 

Multiplier 

mulL) 

Score 

108. 

Presence  of  'critical  environ- 

0  1  3 

1 

3 

ments*  within  1.5  miles  (2.4  km) 
of  the  site 

' 

109 

Sum  of  items  106  through  108 

_ 

21 

110. 

Final  Ecological  Receptors  score 
for  ground  water  pathways  (item 
11091  X  100/21) 

— 

HUMAN  HEALTH  RECEPTORS-AHUSOIL  VOLATH,ES/DUCT  PATHWAYS 

Score 

Product 

Score 

(score  X 

Maximum 

(circle  one) 

Multiplier 

mult.) 

Score 

111. 

Population  within  4  mile  radius 

0  9  12  15 

18  21  24 

1 

— 

30 

27  30 

112. 

Land  use 

0  123 

2 

_ 

6 

113. 

Distance  to  nearest  installation 
boundary 

0  123 

1 

3 

114. 

Sum  of  items  [111]  through  [113] 

— 

39 

115. 

Final  Human  Health  Receptors 
score  for  air  pathways  (item  [114] 

X  100/39) 

ECOLOGICAL  RECEPTORS-AIR/SOIL  VOLATILES/DUST  PATHWAYS 

116. 

Distance  to  sensitive  environment 

0  123 

2 

6 

117. 

Presence  of  'critical  environ¬ 
ments'  within  1.5  mile  (2.4  km) 
of  the  site 

0  3 

1 

— 

3 

118. 

Sum  of  items  [116]  and  [117] 

9 

119. 

Fiiud  Ecological  Receptors  score 
for  air  pathways  (item  [118]  x 
100/9) 

— 
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SCORING  SUMMARY  SHEET 


Pathways 

Score 

Contaminant 
Hazard  Score 

Receptors 

Score 

120. 

Surface  water/human  health  scores 

(  X 

X 

)  /lO.OOO 

item  [12] 

item  [49]/[54] 

item  [93] 

121. 

Surface  water/  ecological  scores 

(  X 

X 

)  /lO.OOO 

item  [12] 

item  [52]/[56] 

item  [97] 

122. 

Ground  water/human  health  scores 

(_  X 

X 

)  /lO.OOO 

item  [22] 

item  [59]/[64] 

item  [105] 

123. 

Ground  water/  ecological  scores 

(_  X 

X 

)  /  lO.OOO 

item  [22] 

item  [62]/[66] 

item  [110] 

124. 

Air/Soil  volatiles  human  score 

(  X 

X 

)  /lO.OOO 

item  [34] 

item  [69] 

item  [115] 

125. 

Air/Soil  volatiles  ecological  scores 

(  X 

X 

)  /lO.OOO 

item  [34] 

item  [72] 

item  [119] 

126. 

Air/Soil  dust  human  health  scores 

(_  X 

X 

) /lO.OOO 

item  [46] 

item  [79] 

item  [115] 

127. 

Air/Soil  dust  ecological  score 

(  X 

X 

)  /lO.OOO 

item  [46] 

item  [82] 

item  [119] 

OVERALL  SITE  SCORE 

In  this 

equation  use  the  higher  of  the  following  pairs  of  values  ([126]  or  [124])  and  ([127]  or  [125]). 

128. 

[(  )'  X  5  +  (  )*  +  (  ) 

*  X  5  +  ( 

)^  +  (  )*  X  5  +  (  )T  = 

item  [120]  item  [121]  item  [122] 

item  [123] 

item  [124]  item  [125] 

or  [126]  or  [127] 

129. 

Over  all  site  score  = 

/4.24  = 

item  [128] 
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Appendix  B:  List  of  Expert  IRP  Managers  Interviewed 


IRP  Managers  that  Participated  in  the  Sample  Test  Case  to 
Rank  Progress 

Sharon  A.  Geil 

Installation  Restoration  Program  Manager 
HQ  AMC/CEVR,  Scott  AFB  IL 

Karl  Kneeling 

Installation  Restoration  Program  Manager 
HQ  USAF/CEVR,  Bolling  AFB  DC 


T.  Wayne  Ratliff 

Installation  Restoration  Program  Manager 
HQ  AFMC/CEVR,  Wright-Patterson  AFB  OH 


Additional  IRP  Managers  Interviewed 
Jeff  K.  Mundey 

Chief,  Environmental  Restoration  Division. 
HQ  AFMC/CEVR,  Wright-Patterson  AFB  OH 


Major  Stuart  A.  Nelson 
DERA  Program  Manager 
HQ  USAF/CEVR,  Bolling  AFB  DC 
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SAMPLE  IRP  SITE  DATA 


Appendix  B:  Sample  Data  Used  for  Personal  Interviews 
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SAMPLE  IRP  SITE  DATA 


Site  ID  -  Fictitious  name  for  sample  IRP  sites  from  which 
actual  DPM  data  was  used. 


PPM  Score  -  DPM  scores  submitted  to  the  Air  Staff  for  the 
sample  sites.  The  sites  were  scored  by  IRP 
managers  at  the  bases. 

Change  in  Risk  Possible  -  This  is  the  amount  the  DPM  score 
could  be  reduced  by  changing  all  of  the  input 
parameters  that  can  be  affected  by  remedial 
actions . 

Change  in  Risk  Achieved  -  This  is  fictitious  data  made  to 
represent  a  variety  of  typical  site  status 
scenarios . 

%  Change  Achieved  -  This  is  the  Change  in  Risk  Achieved 
divided  by  the  Change  in  f^isr.  Possible,  then 
multiplied  by  100  to  get  a  rt^rcentage. 

Site  Closed  -  This  is  fictitious  data  representing  whether 
the  sites  have  been  closed  out  (by  Air  Staff's 
guidance  for  IRP  scorecard) . 


Instructions 


Please  provide  your  expert  opinion  on  how  you  would 
rank  the  20  sites  base  on  what  has  been  accomplished  as  a 
result  of  the  IRP.  Please  order  the  sites  from  1  to  20. 

You  may  use  your  own  personal  opinions  or  beliefs  and  /or 
various  management  strategies  used  to  manage  the  IRP.  I 
would  like  to  have  a  brief  personal  interview  to  discuss  how 
you  weighed  or  considered  each  of  the  factors  to  rank  the 
accomplishments  at  the  sample  sites. 

I  appreciate  your  interest  and  help  in  this  research. 

If  you  have  any  questions  please  contact  me.  My  home  phone 
number  is  (513)  429-1148  or  you  can  leave  a  message  at 
AFIT/DE,  (513) 

255-2156,  DSN  785-2156. 
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Appendix  D;  List  of  Acronyms 


ADI 

ADPM92 

AF 

AI 

CERCLA 

DERA 

DERP 

DIM 

DOD 

DOE 

DPM 

EPA 

FTA 

HARM 

HQ 

HRS 

IRP 

MAJCOM 

MCDM 

MEPAS 

NAS 
NPL 
OSD (E) 


Average  Daily  Intake 

Automated  Defense  Priority  Model,  version  FY  92 
Air  Force 

Artificial  Intelligence 

Comprehensive  Environmental  Response,  Compensation 
and  Liability  Act 

Defense  Environmental  Restoration  Account 

Defense  Environmental  Restoration  Program 

Displaced  Ideal  Model 

Department  of  Defense 

Department  of  Energy 

Defense  Priority  Model 

U.S.  Environmental  Protection  Agency 

Fire  Training  Area 

Hazard  Assessment  Risk  Model 

headquarters 

Hazard  Ranking  System 

Installation  Restoration  Program 

Major  Command 

Multi-Criteria  Decision  Model 

Multimedia  Environmental  Pollutant  Assessment 
System 

National  Academy  of  Sciences 
National  Priority  List 

Office  of  the  Assistant  Secretary  of  Defense 
(Environment ) 
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PA 


Preliminary  Assessment 
QC  Quality  Control 

RI  Remedial  Investigation 

SARA  Superfund  Amendments  and  Reauthorization  Act 

SI  Site  Inspection 

USAF/CEV  U.S.  Air  Force  Civil  Engineering,  Directorate  of 
Environmental  Management 

USAF/CEVR  U.S.  Air  Force  Civil  Engineering,  Directorate  of 

Environmental  Management,  Installation  Restoration 
Division 

UST  Underground  Storage  Tank 
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